embedded linux for purposes of internet appliances

MMAASSTTEERRSS TTHHEESSIISS BBYY EEDDIINN BBEEKK TTEESSEEVVIICC

EEMMBBEEDDDDEEDD LL IINNUUXX FFOORR PPUURRPPOOSSEESS OOFFIINNTTEERRNNEETT AAPPPPLL IIAANNCCEESS

((aa..kk..aa.. TThhee WWaannddeerreerr PPrroojjeecctt))

CCOONNDDUUCCTTEEDD AATT NNAATTIIOONNAALL SSEEMMIICCOONNDDUUCCTTOORR,, SSTTOOCCKK HHOOLLMMSSUUPPEERRVVIISSOORR:: BBEENNGGTT AANNDDEERRSSSSOONN

EEXXAAMMIINNEERR:: PPRROOFF.. GGEERRAALLDD QQ.. MMAAGGUUIIRREE,, PPHHDD

RROOYYAALL IINNSSTTIITTUUTTEE OOFF TTEECCHHNNOOLLOOGGYY ,, SSTTOOCCKK HHOOLLMM -- AANNNNOO 11999999..

EEMMBBEEDDDDEEDD LLIINNUUXX FFOORR PPUURRPPOOSSEESS OOFF IINNTTEERRNNEETT AAPPPPLLIIAANNCCEESS

MMSScc.. tthheessiiss rreeppoorrtt bbyy EEddiinn BBeekktteesseevviicc

~ i ~

AABBSSTTRRAACCTTAs the Internet continues to grow, now new Internet appliances are being marketed. At the sametime, Linux is gradually becoming a competitive choice for desktop, server and embeddedapplications. How do these two facts relate to each other? What should be the position of Linux inInternet appliances, and how interesting is Linux? Where are the Internet appliances reallypositioned in terms of their needs for an operating system? How is Linux implemented inembedded applications?

These are just some of the questions posed during this MSc. Project that this report aims toanswer. In addition, we will examine how Linux is used in embedded applications and what itprovides in terms of achieving their goals. Also, we will take a look at the compression methodsthat can be used for system size reduction and analyze their costs.

The report will show that Linux is a competitive alternative for devices like WebPAD, that itleaves space for further time and cost efficient improvements, and that Internet appliances can getmost of the benefits of Linux during its further evolution relatively cheaply.



~ ii ~

TTAABBLLEE OOFF CCOONNTTEENNTTSS

ABSTRACT I

TABLE OF CONTENTS II

ACKNOWLEDGMENTS VI

1. INTRODUCTION 1

1.1. LAYOUT OF THIS REPORT 1

2. BACKGROUND 2

2.1. WEBPAD - AN OVERVIEW 22.2. WHY WEBPAD? 42.3. WHY L INUX? 5

3. PROBLEM DESCRIPTI ON 6

3.1. SYSTEM PREMISES 7

4. METHOD DESCRIPTION 8

5. LINUX, THE OPERAT ING SYSTEM 9

5.1. INTRODUCTION 95.2. L INUX L ICENSING 105.2.1. GENERAL PUBLIC LICENSE (GPL) 105.3. L INUX IN BRIEF 105.4. SYSTEM ARCHITECTURE 105.5. KERNEL 115.5.1. SPEED OPTIMIZATION 125.5.2. KERNEL MODULES 125.6. BINARY FORMATS 135.6.1. ASSEMBLER OUTPUT (A.OUT) 135.6.2. EXECUTABLE AND LINKABLE FORMAT (ELF) 135.7. STANDARD C LIBRARIES 135.7.1. LIBC4 14



~ iii ~

5.7.2. LIBC5 145.7.3. GLIBC2 145.7.4. CONCLUSION 145.8. X WINDOW SYSTEM 155.8.1. X FEATURES 155.8.2. CONSIDERATIONS AND CONCLUSION 165.9. DEVELOPMENT IN L INUX 16

6. EMBEDDED LINUX 17

6.1. INTRODUCTION 176.2. WHY EMBED L INUX 176.2.1. REAL-TIME EXTENSIONS 186.3. FILESYSTEMS AND DIRECTORY STRUCTURE 206.4. BOOT AND COMPRESSION 206.4.1. BOOTING THE SYSTEM 206.4.2. COMPRESSION 226.5. RUN TIME ISSUES 246.5.1. WHY AND WHEN READ-ONLY? 246.5.2. MODIFIED FILE SYSTEMS 24

7. WWW BROWSER OF CHOICE 26

7.1. INTRODUCTION 267.2. AVAILABLE L INUX BROWSERS 267.2.1. NAVIGATOR 2.02 267.2.2. NAVIGATOR 3.04 277.2.3. NAVIGATOR 4.XX 277.2.4. MOZILLA AND NAVIGATOR 5 TO-BE 287.3. BROWSER CUSTOMIZABILITY 287.3.1. REMOTE CONTROL AND INTER-APPLICATION COMMUNICATION 287.3.2. PLUG-INS, VIEWERS AND PLAYERS 297.4. CONCLUSIONS 30

8. PERSONAL ACCESS DEVICES 32

8.1. RSC WEBPAD 328.2. QUBIT 338.2.1. OUTSTANDING FEATURES 338.2.2. CONCLUSION 34

9. IMPLEMENTATION AND RESULTS 35

9.1. IMPLEMENTATION 359.1.1. EVALUATED MICRO-DISTRIBUTIONS 359.1.2. HARDWARE USED 379.1.3. REQUIRED LIBRARIES AND BINARY FORMAT 379.1.4. SYSTEM SIZE 39



~ iv ~

9.2. SYSTEM DESIGN 419.2.1. SCOPE 419.2.2. INTRODUCTION 429.2.3. RUN-TIME TASKS 429.2.4. INTRODUCING COMPRESSION 439.2.5. SIZE OF COMPRESSED SYSTEMS 449.2.6. CONCLUSION 46

10. CONCLUSIONS 47

10.1. INTRODUCTION 4710.2. EMBEDDED L INUX 4710.2.1. SUPPORT 4710.2.2. SIZE 4810.3. L INUX PADS 4810.3.1. DESIGN ISSUES 4810.3.2. ALTERNATIVES 4910.3.3. CONCLUSION 4910.4. OPPORTUNITY I: WEBPAD LITE BUNDLE FOR PC OWNERS 5010.4.1. SAVING MONEY 5010.4.2. BOOSTING FUNCTIONALITY 5010.4.3. CONCLUSION 5110.5. OPPORTUNITY II: PLUG-INS, LITERALLY SPEAKING 52

11. FUTURE WORK 54

APPENDIX A: LINUX MARKET OVERVIEW 55

MARKET SEGMENTS TARGETED 55MARKET STATISTICS 56ADDITIONAL MARKET INDICATORS 57

APPENDIX B: STORING THE EMBEDDED SOFTWARE 58

INTRODUCTION 58COMPACT FLASH II 58OVERVIEW 58CF II FLASH MEMORY 58IDE 2000 FLASH DISKS 59DISKONCHIP 59IBM MICRODRIVE 60OVERVIEW 60MICRODRIVE FEATURES 60PRICE COMPARISON 61

APPENDIX C: LOCATION TRACKING 63



~ v ~

OVERVIEW 63SOVEREIGN PALTRACK 63PINPOINT 64WHERENET FIREFLY 64APPLICATIONS 64

REFERENCES 65



~ vi ~

AACCKKNNOOWWLLEEDDGGMMEENNTTSS

I am hereby offering my kindest thanks to thefollowing people:

The Linux community and contributors tothe Linux-Embedded and W.A.S.T.E. mailinglist for extensive help;

People at National Semiconductor inStockholm, Bengt Edlund and my supervisorBengt Andersson for making this projectpossible:

My examiner Prof. Gerald Q. Maguire, PhDfor all his advices and patience.

To my family for not giving up on me



~ 1 ~

11.. IINNTTRROODDUUCCTTIIOONNIn the course of the past few years, and as a result of technology shifts and rapid price cuts withinthe hardware industry, a totally new market has now unmistakably emerged: the market formobile computing devices. The wide variety of such devices is offering almost everything one couldexpect from his or her desktop computer, but some even introducing new services. As the abovementioned trend only seems to keep getting even more significant, and since the products of thiskind have already met a very enthusiastic response from their end-users, one can expect thismarket to become even larger and more important for the industry.

To address this trend, last year (1998) at Comdex National Semiconductor (NSC) presented aconcept product, called WebPAD, in order to guide market development and show that it is wayahead its competitors within this field. The WebPAD was an immediate success, and is expectedto be mass-produced in several different forms from its original concept.

In spite of previous attempts to bring mobile devices and the Internet connectivity together (suchas WAP), NSC has chosen a rather unique approach. The WebPAD is intended to be a single-application computing device, used solely for surfing on the Internet. Thus, the WebPAD isthought to open the way for a number of Internet appliances for ordinary peoples homes. Theoperating system used for the proof-of-concept device is QNX with its GUI, Proton, and itscustomized WWW browser, Voyager.

11..11.. LLAAYYOOUUTT OOFF TTHHIISS RREEPPOORRTT

This report is intended as a partial fulfillment of academic requirements for a MSc. projectconducted at Royal Institute of Technology. Moreover, the report attempts to give the reader arather thought-provoking overview of Linux, especially in terms of its implementation inembedded devices. The report is intended to serve as a useful unbiased basis for furtherdiscussions on the subject of development of Internet Appliances, and specifically PersonalAccess Devices for that matter. It should be a practical starting point for individuals interested orinvolved in this puzzle, foremost employees of National and their OEMs.

The layout has been outlined accordingly, and the language used is customized for this purpose.Some elementary knowledge of UNIX and current market trends should be sufficient in order tofollow the reasoning sketched in this paper.

The main focus of this report in terms of describing Linux will be placed on its x86implementation, unless otherwise stated.



~ 2 ~

22.. BBAACCKKGGRROOUUNNDDIn this section we will firstly briefly describe the WebPAD concept as originally presented in [6],[27], [30],[35], [37], [50]. However, while we will address the specifics of this particular conceptdemonstrator, the basic principles shown here are common for a whole line of devices, includingall the Personal Access Devices to be addressed later in this paper.

Secondly, we will outline the background of how this thesis came about.

22..11.. WWEEBBPPAADD -- AANN OOVVEERRVVIIEEWW

The fundamental idea leading to the WebPAD concept is unmistakable to make the Internet asunconstrained and convenient to use for a consumer, as a plain cordless telephone. Theprevailing goal set for the concept behind the WebPAD is to bring Internet into ordinary peopleshomes without the inconvenience almost inevitably accompanying PC desktop computers, thecurrent most likely tool for accessing the Web. This goal is to be accomplished through localwireless connectivity, as one of the concepts main design characteristics.

With this portable, always-on, hand-held tablet weighing less than three pounds, the homeuser is given the ability to explore the Web and still move quite freely. The usual input devicessuch as keyboard or mouse are substituted with an intuitive touch screen interface and a soft, on-screen keyboard. The only software running on the WebPAD visible to the user is the webbrowser.

Naturally, the entire package is intended to have a low price, or even be handed out for free byInternet Service Providers (ISPs). As this marketing concept has proven to work well in case ofe.g. cellular phones, it would make it possible for a critical mass of WebPAD users to be reachedswiftly.

Figure 2-1. WebPAD tablet (With permission of NSC)



~ 3 ~

A generic WebPAD system consists of three parts:

The tablet itself (see Figure 2-1)

The base station transceiver, as a gateway to the Internet

The charging stand/docking station

Optional extensions might include a wireless or USB: keyboard,mouse, printer, etc.

The WebPAD tablet comprises:

x86 architecture, processor based around the low-power version ofthe National Geode GXLV1 microprocessor. The high level ofintegration used in these processors results in: small size, low costand easy expandability of the hardware core.

Radio interface. Either Wireless LAN or DECT, used forinterconnecting the device to the Internet gateway.

10.4 TFT screen with a touch screen.

QNX embedded diskless operating system, including Photon andVoyager. The OS features real-time, while the browser has supportfor JavaScript. However, the browser is currently not Java-enabled,and the number of plug-ins and other extensions is very limited.

Flash storage (16 MB)

Multimedia capability. Stereo speakers and a microphone, withSoundBlaster 16 compatible integrated audio.

USB port

Figure 2-2 recapitulates the supplementary features of the concept (With permission of NSC)

1 This is the new name for the MediaGX CPU.



~ 4 ~

As one can see in Figure 2-2, the tablet should be able to use a set top box, a desktop PC or astandalone base station as a gateway to the Internet. Accordingly, the further connection to theInternet is by no means limited by the concept and could be achieved via a dial-up modem or apersistent cable link.

The communications between the WebPAD tablet and the gateway is achieved through radiotransmission. For this purpose, the initial version of the WebPAD uses Wireless LAN based onthe Harris PRISM chipset. Moreover, an additional solution has been presented, using DECTinstead, presumably allowing a significantly lower total price while still retaining reasonablyhigh throughput.

The WebPAD is meant to be used for [6]:

Information access (movie show times, daily news, recipes)

eCommerce (on-line purchasing)

Communication (email, voice, chat)

Games (on-line entertainment)

Financial (stock quotes, on-line banking)

In addition, the tablets design and potential extendibility is described this way [6]: The WebPADis completely integrated and sealed: There are no internal parts that can be upgraded, added or removed,and there is no software to load. It's designed with dual Universal System Bus (USB) ports to addperipheral options such as a keyboard, mouse, printer or gaming input device. There are no disk or floppydrives, and no PC Card slots -- although the design would support these items if an OEM chose to featurethem.

22..22.. WWHHYY WWEEBBPPAADD??

WebPAD is intended to be used by people who do not have the necessary level of knowledge tooperate a PC, but nevertheless want to be able to surf on the Internet2. Additionally, it can beused by people who just want a more convenient way of accomplishing the same goal, regardlessof their IT literacy. It is important to notice that the WebPAD is not meant to replace conventionalPC the home user might already own. As a gaming or a productivity tool, the PC is stillconsidered the only appropriate solution for the masses of consumers [35].

So, are they being produced? Not yet. By developing the WebPAD, the primary intention of NSCis not to change the market, but to show that its integration strategy gives Original EquipmentManufacturers (OEMs) the right means and a lot of freedom when designing consumer products.OEMs themselves should design their products, customizing the original concept to their marketsegment.

Therefore, the conceptual design has been provided to interested OEMs as a reference design.They will customize the original concept to their needs and market their systems accordingly.Two companies that have publicly announced their intentions to introduce their variants of theoriginal WebPAD concept are Tatung and RSC.

2 Or as Todd Beecher, manager of strategic marketing for Conceptual Products, the division that designed the WebPAD,explains it: We took a chance. I guess my focus group was my grandmother..



~ 5 ~

22..33.. WWHHYY LLIINNUUXX??

The target customer is, as we saw, principally the home user, who is given the opportunity toaccess the World Wide Web in a wireless manner whilst moving around in their home, using thiscomputerized tablet. A question that still remains to be answered is how interesting the conceptis for the corporate user, an enormous market potentially to be targeted subsequently byinterested OEMs, given that the initial effort succeeds. The general conceptual design supports aclient-server architecture already, but there is still an open question of potential business-orientedapplications.

At issue in both of those major application areas was the question of: what operating systemshould be used? In other words, what operating system could provide the home user with an easy-to-use, easily configurable interface, while still allowing power-applications for a corporate user?At the same time, the development costs for the OEMs indirectly dictate the price and theprobability of success of such a venture, and should therefore be as low as possible. One singledesign characteristic of such a system is certain: it should be widely used, as well as easilyscalable and customized. At the same time, its price should be low enough to allow for areasonable priced final product. Moreover, it should be widely known and used, so that it cangrow with its newfound market and provide technical solutions for the services to beencountered in the future.

The operating system should by no means be regarded as a goal on its own, but as a means ofproviding a good user interface and a useful, cost efficient, up-to-date product. One has to beaware of the fact that the web browser chosen for the purposes of WebPAD-alike appliances isresponsible for the ultimate look and feel of the system as a whole, and the OS should remaintransparent. However, the choice of browser is tightly coupled to the choice of OS, due to thesimple fact that not all browsers are implemented on all major operating systems. For example,QNX operating systems are supported in terms of browsers only internally, by QNX itself, whoare trying to follow the market trends without much help from outside. This also means that theymust develop their own plug-ins, viewers/players for major widely used formats, such asAdobes Acrobat PDF, Real Networks Real Audio/Real Video, Macromedias Shockwave, etc.

Having all of these major demands on the system in one's mind, one comes up with a simplealternative worth evaluating, Linux. Linux, the operating system available free of charge. Linux,the operating system widely used and with an enormous base due to its ancestors, variousflavors of UNIX. Linux, the operating system with the fastest growing market share in the worldand an impassioned community behind it. Linux, which major corporate solutions providers,such as IBM, Compaq, Informix, Oracle and SyBase have already decided to fully support.



~ 6 ~

33.. PPRROOBBLLEEMM DDEESSCCRRIIPPTTIIOONNInternet appliances of the WebPAD variety rely heavily on continuously evolving Internettechnologies. The final version of the WebPAD is to be a consumer product.

Let us now consider some more specific issues and present limitations of the current WebPADthat a critic is sure to state. For the time being, the operating system running on the WebPAD isQNX in combination with its GUI Proton and its browser Voyager. Although this OS is wellknown for its small footprint, which can be regarded as one of the crucial attributes of a classicalembedded system, there are a number of downsides of which only the most important ones willbe listed here:

It is a commercial, pay-per-license, product. Although its price mightbe relatively low, because a WebPAD-like consumer product wouldtarget entry-level computer market, price raises an important issue.Its strong commercial heritage also draws additionalcopyright/secrecy issues concerning the work done and sharedwithin the company, in contacts with partners and the end-users.

QNX is an OS that is regarded relatively exotic among developers. This inturn leads to the fact that software products developed outside QNXitself and offering acceptable quality and stability are very few innumber and are naturally commercial, expensive products.Furthermore, it is comparably very difficult to find and hireseasoned professionals experienced in QNX development.

The QNX web browser, Voyager, is rather basic when compared to thepresent Internet standards. It is not Java-enabled, and the plug-ins andviewers/players for even the most popular web formats andservices are still at the development stage. Worse is that it is highlyprobable that this unfortunate state is going to remain. QNX issimply not considered important- enough to support by majorsoftware providers. QNX themselves, as already mentioned, areworking on delivering support for Java and some major plug-informats. The quality and stability of these new add-ons is yet to beproven in practice. Its development pace is, compared to majorbrowsers, very slow.

As Linux is the proposed alternative and prior to the deeper discussion in this report, thefollowing questions arise:

How big is its footprint? Linux wasnt originally intended forembedded applications, neither were its major browsers and its GUI,X Windows, although the latter has been used quite frequently inconjunction with X terminals. Thereby the size constraints haventbeen taken as seriously during their development, as in the case ofe.g. QNX, although X terminals are quite commonly used.

How well does Linux fit the needs of embedded applications? Although ithas taken a noticeable share of the OS market in the desktop andserver segment, Linux is still not as natural a choice for embeddedsolutions, as some operating systems developed for this purpose.



~ 7 ~

Although the amount of software available for Linux might be relativelylarge, does it mean much in terms of embedded development? Manydecision-makers wonder if all that free software can actually beutilized.

Let us assume for a second that Linux is an appropriate choice, which we will substantiate in thechapter on Embedded Linux. Now, a third set of questions arises:

How does one implement Linux on Information appliances like theWebPAD?

Would Linux, in such a case, offer any new opportunities or easyincorporation of known ideas?

What web-browser should be used? What device drivers can already befound out there, and which ones need writing/porting?

These issues, among others will be discussed in the rest of this document.

33..11.. SSYYSSTTEEMM PPRREEMMIISSEESS

The run-time Linux WebPAD software should consist of following main elements:

Essential Linux (kernel, libraries etc.)

Necessary daemons and services (apmd, pcmcia), as well asdrivers

Minimal X Windows subsystem (X server, only vital Xlibraries, fonts, etc.)

Browser and additional components (binary executable, Javaclasses, additional shared libraries, etc.)

Container application, yet to be written. This applicationplays the role of the visible WebPAD GUI.

Soft keyboard application, yet to be written. The Virtualkeyboard was described as a pop-up keyboard and wasdeveloped as part of the IBM-Columbia University StudentNotebook Project in 1989 [7]. In our application it is used forentering URLs and such.



~ 8 ~

44.. MMEETTHHOODD DDEESSCCRRIIPPTTIIOONNThe main goal of the project was to evaluate Linux in terms of the WebPAD and implement thesystem as far as possible. During the evaluation part the author took Linux into considerationfrom mainly a technical and, to some extent, marketing perspective in order to determine whatthe WebPAD running Linux should look like and what features it would and should have. Theimplementation part then focused on proving and testing the major conclusions from theprevious stage.

In order to evaluate, and then implement a Linux-based solution on WebPAD, one should studyat least the following issues of crucial importance for this project:

The Linux footprint. Linux is not intended to be as small as QNX, but it offersmany more components. The question is how many and which componentsshould be implemented, especially since there are strong dependenciesamong most of them. Importantly, since it most probably will be stored inthe quite expensive flash memory, it should be kept as small as feasible. Thisis stressed as the definitely most important single issue to consider.

The price and functionality of a potential Linux solution, and WebPAD in general.As we saw earlier, the WebPAD is targeting quite low price points, and it istherefore extremely important to provide a product with a goodprice/functionality ratio.

Device drivers and supplemental software. Although there are a large number ofdevice drivers for Linux available, the technology the WebPAD is based onis very new or uncommon (e.g. touch screen), which may mean that some ofcritical drivers either are not released for Linux, or are not stable yet. Theabove named problem was unfortunately underlined by the fact that only afew hardware manufacturers offer first-hand Linux support starting fromthe initial release stage of their products. Therefore most drivers aredeveloped independently by the enormous Linux community. In addition,one should investigate if there is software needed for WebPAD-like devices,already developed, or modifiable with a reasonable effort.

Web browser. What browser should be chosen? Netscape itself is actually a setof multiple solutions, since the latest version does not necessarily have to bechosen. One might prefer e.g. versions 2.x and 3.x since they provide most ofthe functionality, with a considerably smaller memory footprint.Additionally, some versions of Netscape browsers are available in a fewforms, requiring various kernel, library and plug-in compatibilitycombinations. At the top of the list, there is the browser-only option(Navigator), as well as a suit of TCP/IP applications (Communicator)incorporating e-mail and news clients. On the other hand, a stripped downMozilla might be preferred. Other browsers available for Linux should belooked into as well.



~ 9 ~

55.. LLIINNUUXX,, TTHHEE OOPPEERRAATTIINNGG SSYYSSTTEEMM

55..11.. IINNTTRROODDUUCCTTIIOONN

Linux got started when a broke Finnish student, Linus Torvalds, bought his first 386 PC computerback in the early 90s, and declined to pay for a copy of the Microsoft Windows OS. Instead, hepurchased a cheaper version of, even for that time odd, minix x86 operating system. Because hewas not satisfied with this Windows alternative either, inspired by [42]3 he started developing akernel of his own, intending to implement some basic POSIX functionality and see whathappened. Upon reaching the initial milestone of this, at the time hobby project, he posted hisfirst alpha release on the Usenet. The rest is history.

Ever since, Linux has evolved into something much bigger than even the most optimisticenthusiasts could have pictured in their wildest dreams. A tremendous number of supportersacross the world joined, successively developing additional extensions for the kernel that startedthe avalanche. The UNIX community also came to endorse the project by providing continuouslyimproved support for some of the main applications of a UNIX system; including but not limitedto the cursed and adored X Windows GUI, suites of compilers, networking utilities andapplications, development environments, etc. At the same time, entirely new andgroundbreaking applications were being created ceaselessly by the Linux community, solely forLinux.

This operating system is now believed to endanger Microsofts vision of the future server marketentirely dominated by NT and its successors, and is seen by some as the fittest contender offeredby the massive UNIX community. The Linux penetration in the desktop market segment has sofar been described as, geeks with too much time on their hands. However, since this OSentered a very comfortable symbiosis with the commercial world, a lot has radically improvedeven in this market segment. Major Linux distributors are being introduced to the stock exchangefollowed by almost tireless media coverage, as major solution providers have decided to supportLinux as one of their reference platforms. Meanwhile, the Linux community is continuing towork on providing solutions that would make it possible even for total beginners to install Linuxon their desktop PCs and use it efficiently on a daily basis.

To this point, Linux has shyly been introduced to the embedded market as well. Indeed, since theembedded market tends to implement more and more of the functionality so far reserved for thedesktop solutions, including the WebPAD concept for example, it definitely has a rather brightfuture even there. The argument regarding embedded Linux will be continued in the followingchapter.

In this chapter, we will give an overall description of this operating system, addressing evenissues that will be of crucial importance in the further analysis. The very extensive documentspresented in [43], here sporadically used as a quick reference, are recommended to the interestedreader an a source of further information about Linux itself.

The reader not familiar with the Linux market situation is advised to firstly read Appendix A:Linux Market overview.

3 As a bizarre coincidence, one might note that the author unknowingly used the same source in the preparation of thisMSc. project. However, Linus can consider himself lucky at one point: All he had to do was create one of the majoroperating systems in the world; not try to write a report on it.



~ 10 ~

55..22.. LLIINNUUXX LLIICCEENNSSIINNGG

As opposed to the rather common misconception, Linux is not distributed in the public domain.The other commonly known licensing concepts like shareware do not apply to Linux either. Theterm freeware is probably the one that describes it in the most accurate fashion, but even this termis unfortunately not precise enough.

55..22..11.. GGeenneerraall PPuubblliicc LLiicceennssee ((GGPPLL))Linux kernel and most of the accompanying software packages are released under the GNUGeneral Public License (GPL)4. This license was written by the Free Software Foundation (FSF). FSFwas started by free software visionary, Richard Stallman.

In layman terms the license means that anybody can use, modify and amend the original sourcecode. The code altered in such a manner can be freely distributed and even charged for.However, if a commercial Linux software package is based on, or contains source code licensedunder GPL, the modified code should be released under GPL as well.

All source code of the system is publicly available and the accompanying packages are freelyavailable. Most of the commercial-quality software products are also free, most often releasedunder GPL as well. However, there are also an increasing number of pure commercialapplications, complying with the usual commercial licensing procedures.

55..33.. LLIINNUUXX IINN BBRRIIEEFF

Linux is essentially an exceptionally stable and robust multi-user, multitasking, UNIX-likeoperating system. It has been ported to a number of microprocessor architectures, including SunSPARC, DEC Alpha, ARM, MIPS, Motorola 68k, PowerPC, etc., and is currently being ported to anumber of others. However, its origin was on the x86 family of processors, on which it runs in the3865 protected mode.

It is UNIX-like due to the fact that its maintainers advisedly chose to follow the POSIXspecification, additionally incorporating even System V and BSD extensions. Therefore, it isgenerally compatible with these systems at the source level.

55..44.. SSYYSSTTEEMM AARRCCHHIITTEECCTTUURREE

In the following summary description of Linux components, we will primarily consider Linuxfrom a standpoint relative to its implementation on embedded systems described later in thispaper.

In those terms, the following Figure 5-1 is given as a reminder of the overall Linux structure (seenfrom the above-mentioned perspective). By standard libraries, we assume standard C libraries,

4 Can be viewed at http://www.gnu.org/copyleft/gpl.html

5 However, it can be easily optimized for speed for successive architectures (486, Pentium etc.). Consult page 12 for furtherinformation on this.



~ 11 ~

which are to be briefly described later. Standard programs are often called UNIX commands,such as ls , cat etc. Application libraries are the external libraries used by a certain application6externally from the operating system itself.

Figure 5-1. Linux overall structure

55..55.. KKEERRNNEELL

Linux kernel is the heart of the system. Essentially, only the kernel itself is pure Linux, whilemost of the extensions originate in the UNIX world and GNU, or are designed according to thewell-proven UNIX structure. This kernel is a result of a continuing work by the Linuxcommunity, whose members are distributed across the world. However, the entire kernel projectis monitored and maintained by Linus Torvalds, its original founder. The work is carried out intask teams, each managed by a maintainer. The source code is freely available and distributedunder GPL. The kernel is continuously updated, and even development versions7 are available toeverybody. Since the Linux community of users can at this point be deemed as comparativelyvery qualified, the kernel thus has a vast number of very competent beta-testers.

The kernel is time sharing and non-preemptive. Its source of inspiration is basically the knownUNIX systems, but with a broad set of modern features introduced. As Linux is increasingly usedin academia and within the industry, it supports and is used for a number of state-of-the-artscientific, security and standardization projects. Thus, it has even included support for e.g. IP6and HAM/Packet Radio, for a quite while now. However, it doesnt stop there. For illustrationpurposes, here we will list just a few of the features, easily customizable already when compiling

6 To keep it simple, even X Window System is here fit into this category.

7 Kernels with odd subversion numbers are classified as development, unstable versions. For example, kernel 2.3.x is anunstable, development version of 2.2.x, while 2.2.x is released and stable.



~ 12 ~

your own kernel8, used for optimizing the kernel for speed. Subsequently, we will outline the listof kernel modules. The kernel documentation is sporadically referenced.

55..55..11.. SSppeeeedd ooppttiimmiizzaattiioonnThe speed optimization9 is naturally aimed at improving the kernel performance on a specificprocessor family, belonging to the x86 architecture in the case of the present study. The x86 Linuxkernel classifies the processors in the following five groups:

386 the 386 (including 486SX) CPUs (compilation mode compatible with all other groups).

486 486DX and above, including e.g. Cyrix 5x86.

586 Pentium-class CPUs, potentially without the TSC register10.

Pentium Intels Pentium (alt. MMX) and AMD K5/K6/K6-3D.

PPro all the subsequent CPUs, including Intel P II, P III, Xeon and NSC/Cyrix 6x86MX/M IIprocessors.

Additionally, most of the Linux software packages can be optimized for performance whencompiled, in a similar manner. For example, Mandrake11 claims that they achieve 5-30%performance improvement, by optimizing their distribution for Pentium-class processors.However high this optimization really is, it is certainly of significance, especially in constrainedembedded applications.

The kernel also fully utilizes Memory Type range Register12 in the newer processors, which can helpboost the graphics performance up to 2.5 times in some systems.

Naturally, the kernel also supports Symmetric Processing, allowing for efficient (parallel) use ofmore than one processor in a system.

55..55..22.. KKeerrnneell mmoodduulleessA kernel module is a piece of compiled code that can be dynamically attached to or removedfrom a running kernel, without needing to restart the system. As described in [38], the main goalof modules is to extend the kernels own functionality. The module runs in the same addressspace as the kernel to which it is attached and in the same thread, usually even as a singleprocess.

A module can be virtually anything13 that the kernel doesnt necessarily need all the time, butshould be loaded on demand; thus, saving memory and other resources. The natural

8 The latest kernel was versioned 2.2.12 at the time of writing. It is assumed throughout this report.

9 Originates from the gcc optimization features. Optimization is easily achieved by e.g. using m486 option for 486systems.

10 MediaGX/Geode is said to work most efficiently with this optimization.

11 Mandrake is one of the market-leading Linux commercial distributors.

12 For Cyrix processors, MTRR is successfully implemented by utilizing their Address Range Registers.

13 For instance, automounter kernel feature, mounting filesystems automatically when filesystems are accessed can also becompiled as a module.



~ 13 ~

implementation of classic device drivers is just one of possibilities, that includes even filesystems,support for binary formats, etc.

55..66.. BBIINNAARRYY FFOORRMMAATTSS

Executables and libraries on Linux presently can be built using two14 different binary formats, theolder a.out, and the current standard, ELF.

55..66..11.. AAsssseemmbblleerr OOuuttppuutt ((aa..oouutt))A.out is a set of formats that were used in all the UNIX-like operating systems since thebeginning. In Linux it can be found in two forms, namely ZMAGIC and QMAGIC.

In order to run a.out applications, the kernel has to be precompiled with a.out internal supportwhich is notably not common in the latest kernels. Naturally, the support can be even compiledas a loadable kernel module. Furthermore, a.out support compatibility libraries have to beinstalled.

55..66..22.. EExxeeccuuttaabbllee aanndd LLiinnkkaabbllee FFoorrmmaatt ((EELLFF))ELF is the current standard binary format for Linux, but also for some of the major UNIX flavors,including e.g. Solaris, IRIX and Digital UNIX. As explained in [44], it is more flexible than itspredecessor, a.out, which is one of the main properties, which caused even the Linux communityto shift from a.out to ELF.

In conjunction with the kernel, it is important to notice that the kernel naturally has to beprecompiled with ELF capability on order to run ELF executables.

55..77.. SSTTAANNDDAARRDD CC LLIIBBRRAARRIIEESS

Libraries are files containing routines used by dynamically compiled programs. The libraries areused by software applications as a shared toolbox, thereby reducing the size of programs usingthem. In this section we first examine the standard C libraries, that together with the kerneldefine the framework of every UNIX-like operating system. In order to standardize thisframework, IEEE promulgated the POSIX.1 standard specification, which is widely used whenimplementing and testing UNIX-like operating systems and their extensions.

The Linux libraries contributed have been courtesy of GNU project from the very beginning ofLinux breakthrough. GNU supplied an open source, free implementation facilitating the furtherLinux development. The initial libraries, customarily addressed as libc1, were a derivation ofGNUs larger set of routines. However, over time, in order to get even closer to the POSIX.1standard, the libraries gradually evolved, resulting in consecutive releases. One should note thatprograms compiled with one version of libraries cannot be executed on systems with a differentlibrary release.

14 Misc and Java binary formats are not important in this context.



~ 14 ~

The fact that the GNU C libraries already are ported to many platforms makes portingapplications to/from various UNIX flavors much easier.

55..77..11.. LLiibbcc44Libc4 libraries are the final release of standard C libraries used in conjunction with the old a.outexecutables. Normally, they are not used anymore, since the Linux community has moved on toELF.

55..77..22.. LLiibbcc55Libc5 is the first library version used exclusively with ELF. The main advantage of libc5,compared to libc4 and earlier, results from the ELF compliance itself. Now the libraries could beshared between running processes in a much easier fashion, thereby radically improving systemperformance. Libc5 is still considered to be somewhat of an official standard, although most ofthe major Linux distributions do not use it any more, except for compatibility reasons.

55..77..33.. GGlliibbcc22The newer and currently widely accepted glibc215 libraries were released by the GNU project. Interms of their implementation on space-critical (embedded) systems, one should bear in mindthat they are generally larger than the libc5 libraries. However, as some of the most importantadvantages of using glibc2, even in the embedded systems, the following reasons are given [18]:

Much better conformance with existing standards, i.e. POSIX.116

Support for 64-bit data access, the next boundary in prevailing development, from the very start

Much cleaner structure and implementation, allowing for easier modifications and extendibility,as well as improved security

Included support for IP 6

Name Service Switch, making changing database access methods possible, allowing processesto access system information

Additional support for internationalization

Portability routines, allowing easier optimization for particular hardware architectures

Improved multithreading support

Symbol versioning, for compatibility with library versions to come

55..77..44.. CCoonncclluussiioonnSumming up, one can stress that glibc2 is in general definitely the standard to follow, unless thereare especially good reasons against using it in a particular situation. The cases where one shouldconsider using libc5 instead are characterized by utter space criticality, however this means thatthe product is not likely to easily evolve to use the next stage of Linux development.

15 Glibc2 is occasionally also called libc6.

16 Optionally implements POSIX 1003.1c pthread interface.



~ 15 ~

55..88.. XX WWIINNDDOOWW SSYYSSTTEEMM

X Window System has been the classical GUI of almost all UNIX-like operating systems for acouple of decades now. Although some see its age as one of its main disadvantages, it has alwaysbeen the test framework for groundbreaking development in the GUI sphere. Over time itbecame very stable and has evolved into a modern GUI.

55..88..11.. XX ffeeaattuurreessX Window System essentially implements a TCP/IP-based client-server model, according towhich clients, i.e. X applications wanting to show or modify something on the screen, issueprotocol requests to the X server that in turn executes these requests on the graphics hardware.

However, unlike e.g. Windows 9x/NT, it is not a monolithic system. For instance, managingwindows (raising, minimizing, moving etc.) is handled by an additional and easily exchangeablecomponent, the window manager, while the buttons, icons etc., are distributed by various GUItoolkits. Therefore, it is quite easy to change both the look and feel of the GUI.

As its modular design approach certainly has significant advantages such as the freedom toutilize whatever alternative one prefers, it also leads to different applications providing quitedifferent, non-standardized, user experience and application behavior. In terms of embedding X,this also implies that different toolkits are likely to be used in various X applications, consumingspace. The former fact has been seen as one of the obstacles for Linux, to make a seriousbreakthrough on the desktop market and appeal to common users. In order to provide a unifiedand standardized solution, Gnome17 and KDE18 projects have been initiated, and they are alreadystable and now commonly used releases delivering creative, inventive features.

There are several commercial X window suites for Linux such as Accelerated X and Metro X19.However, the most commonly used is a freely redistributable X/Open standard compliant suitedeveloped by XFree86 Project20. Naturally, even the source code is freely available. The Linuxcommunity and two of the major Linux distributors (Red Hat and SuSE) are some of its maincontributors.

It is based on X11R6.3 specification21 and considered very stable. X servers themselves are, inturn, very hardware dependent. This means that there is a separate X server for almost everymajor graphic chip22, while the rest of the system is common. This has proven to be one of themain obstacles on the road to desktop users.

17 A quite different avant-garde desktop and window platform, with an astonishing touch and feel and level ofcustomization.

18 A clean-cut, extensive desktop platform, seemingly aiming at and very likely to appeal to current Windows 9x/NTusers.

19 The latter has apparently moved on to supporting and contributing to XFree86 instead.

20 More info at http://www.xfree86.org/

21 However, a move to X11R6.4 is planned.

22 MediaGX/Geode is supported by the SVGA X server, starting from a quite recent 3.3.x release, and has proven stableeven during this project.



~ 16 ~

55..88..22.. CCoonnssiiddeerraattiioonnss aanndd ccoonncclluussiioonnThe X Window System and X servers specifically are, relatively speaking23, tremendouslymassive in terms of space they take up. Here, especially the newer releases grew relatively large,helped by the fact that they are available only for libc5 and glibc2 systems24. However, theimmense number of applications written for X, the enormous number of developers raised with(or on) it, as well as its stability, market position and prospects, are definitely on its side25.

55..99.. DDEEVVEELLOOPPMMEENNTT IINN LLIINNUUXX

Since Linux is a UNIX variant, all of the significant development languages and environmentsknown from other UNIX systems have been ported to Linux, including those made by GNU.These tools are of very high quality.

However, this extensive and broad spectrum of tools is soon to be expanded further, since thefirst commercial tools from the Microsoft world are now being brought to the light of the day.The common languages (C, C++, Java, Pascal, Forth, Perl etc.) and compilers/developmentenvironments (gcc, egcs , JDK and so on) are thus now to be supplemented by e.g.Borland/Inprises Delphi, C++ Builder, Jbuilder and VisiBroker26. Metroworks CodeWarrior follows inthe same steps. The producers of the market-dominant database solutions, such as Oracle,Informix and Sybase thereby started a very positive trend.

23 For instance, QNX has in a very successful promotional maneuver managed to put their microkernel, essentialMicroGUI, a simple but efficient WWW browser, as well as TCP/IP stack, generic modem drivers and Internet dial-up ona single, compressed floppy disk. Storing even compressed X Windows alone on a floppy would be difficult.

24 To our best knowledge, the modern design makes utilizing older libc4, a.out libraries impossible.

25 Some potential opportunities that the X window System could open, will be discussed in the Conclusions chapter.

26 VisiBroker and Jbuilder Java 2 are already available at this time. The other Borland products will follow shortly.



~ 17 ~

66.. EEMMBBEEDDDDEEDD LLIINNUUXX


Due to the features of Linux, e.g combined networking, GUI and Symmetric Processing on arange of processors, embedding it is becoming quite common. Additionally, real-time extensionshave been developed.

The support for embedded Linux is becoming solid and stable. Now consulting and industrialdesign companies are broadening their competence to cover Linux as well, which has also lead toupcoming and strengthening of players providing solely Linux services and embedded products.Among these companies, e.g. Zentropix27 and Synergy Microsystems28 specialize in real-time Linux,while IGEL29 provides Linux-based thin client solutions.

This emerging market30 has brought commercial Linux distributors to it as well. Hence, Caldera-owned Lineo31 has announced a road map for embedded Linux. The solutions include Embedix, aLinux-based embedded OS, a supplementary SDK and Embrowser, an embeddable micro-browserintended for use in TV set-top boxes, network computers, information kiosks etc.Linux/Embedix version of this HTML 3.2-compliant browser is expected to ship in the firstquarter 2000. Moreover, Lineo and MeterNet have already announced cooperation on the first set-top box in the market running Linux32.

In this chapter we will consider the most important issues of embedded Linux.

66..22.. WWHHYY EEMMBBEEDD LLIINNUUXX

When the issue of embedding Linux is discussed, many see the fact of Linux being a freeoperating system as its main advantage towards its competitors. Although this property of Linuxand its greatly liberal General Public License (GPL) are important, it is far from being the onlyjustification for implementing Linux in embedded applications. As we will see in this chapter, itis not always even the most substantial and important reason.

With the trend of increasing use of PC-like hardware, Linux with its strong PC-backgroundbecomes a very feasible solution for many practical implementations. As correctly described in[11]33, the alternatives such as DOS and desktop Windows operating systems are very limited,

27 http://www.zentropix.com/

28 Further specializing in PowerPPC. More info at http://www.synergymicro.com/

29 http://www.igel.de/

30 For instance, on their new Wireless LAN thin client series Aero 8000 (scheduled for Q4 1999), Compaq is to introducetwo models in parallel, one running Linux and the other comprising Windows CE.

31 More information at http://www.lineo.com/

32 Interestingly enough, this set-top box is also powered by MediaGX/Geode.

33 Although written in 1997, this article gives an excellent overview of current embedded Linux issues and can berecommended to the interested readers. It should also be consulted when describing real-time extensions later in this text.



~ 18 ~

while expensive, proprietary, and mostly non-portable high-end real-time operating systems cannotefficiently match the needs introduced by this technology shift. Additionally, What is needed is anOS for which support is available-one that is cheap, mature, and provides the features that high-endembedded systems must have, meaning good GUI and networking capabilities. Linux fits thisdescription more than well.

The non-restricted availability of the source code is of high importance in most embeddedsystems, for a number of reasons. For instance, [48] quotes the fact that hardware manufacturerscan share early-access hardware, plus prototyping or proof-of-concept software, with customers and/orpartners without any concerns about software licensing, thus speeding up the entire developmentcycle, as one of the main reasons why Motorola Computing Group was at the time consideringproviding Linux board and system support for their customers and partners34. On the other hand,a case of implementing Linux for control of transportation equipment such as elevators, asdescribed in [26], availability of the source code is crucial for safety-sensitive systems with a longlifecycle.

Other reasons, such as code portability, system stability and good kernel support for watchdog-timers etc., are nevertheless momentous, even in communications systems [15].

66..22..11.. RReeaall--ttiimmee eexxtteennssiioonnssLinux is not a classical real-time operating system. However, due to the fact that many embeddedapplications35 are real-time critical, an extensive amount of work within the Linux community,aimed at providing a real-time Linux, has been put down. This work took an innovativeapproach and led to some very interesting results.

66..22..11..11.. RReeaall--ttiimmee ddeeffiinniittiioonn

A real-time system is, as defined in [2]36, a system that performs its functions and responds toexternal, asynchronous events within a specified amount of time. One of the main characteristics ofsuch a system is that it can guarantee when and for how long a certain process is going to beexecuted, i.e. be given access to the processor(s). Another difference is the fact that non-real-timeoperating systems tend to favor good average performance of processes running, as opposed totiming issues of each and every process as in the case of real-time operating systems.

Real-time operating systems are usually divided in two groups, on basis of grade of real-timerequirements fulfillment:

Soft real-time systems. These are characterized by their timing obligations being definedstatically. Simplified, an occasional flaw is not critical and of too much concern37.

Hard real-time systems. In such systems, processes must be ran as promised, and deadlinesmust be guaranteed38. As stated in [2], Hard real-time systems cannot use average caseperformance to compensate for worst case performance.



~ 19 ~

66..22..11..22.. FFrroomm LLiinnuuxx ttoo rreeaall--ttiimmee

The Linux kernel is, as most UNIX systems, based on time, non-preemptive scheduling, thus isinitially not real-time compliant. Its scheduler is designed to have a solid, balanced response timeand processing throughput in order to provide fair distribution of processor usage time slicesamong all running processes. These processes are assigned different priority levels depending onthe processes track record, in terms of time they used up, how input/output sensitive they are,etc.

For example, if a process doesnt use up all its dedicated processor time, its priority remains thesame or is incremented. Accordingly, if it consumes all of the time it was scheduled, its prioritylevel is reduced. Kernels non-preemptive attribute leads to an additional problem, manifested byhigh priority processes not being able to execute exactly when needed, until the currentlyrunning, low priority process is through. Additionally, software interrupts are disabled as thekernel is running, so e.g. I/O processes demanding instantaneous response are handicapped interms of synchronization. An additional problem is the implementation of virtual memory,adding yet another unpredictability factor.

The kernels performance is therefore poor in respect to real-time demands and is deemed quiteunpredictable. However, real-time extensions have been made, so that an extended kernelbehaves as a classical real-time kernel.

[54], [55] describe the additional reasons for making Linux real-time, and present the extensionsto Linux making it a hard real-time operating system. The design principle is quite simple: theLinux kernel is just moved down in the tree, replaced by the small RT Linux39 kernel, that isrunning the original Linux kernel as a low priority task. All interrupts are thus handled by the RTkernel, and are dispatched to the original Linux kernel when there is no other real-time criticalprocess to run.

As the result of this clever approach, future kernel development of the original, standard Linuxkernel doesnt require an update of the RT system, which would otherwise be almost impossibleto do frequently, due to the kernels complicated nature40. As a result, one has two domains, onereal-rime implementing its essential functionality, and one non real-time where ordinary, non-real-time demanding processes run.

34 As we previously noted, Motorola has now officially announced this step.

35 Assembly lines, various instruments, telecommunications systems, etc.

36 This work, outlined by Barabanov and previously Yodaiken, is the basis of development of now quite extensive real-time support for Linux.

37 Barabanov uses a quite descriptive example: In a real-time video conferencing system, missing frames is naturally notdesirable, but missing one or two occasionally is not critical for the user perception of the distributed message. Forinstance, one sets up a rule, such as: The routine should be executed properly within 100 ms, in 99% of cases.

38 Similarly, in a real-time system controlling behavior of nuclear rocket launcher, shutting it down in case of e.g.overheating simply must be completed in time. As a rule one requires, for instance: The routine is to be executed within 100ms every single time.

39 RT Linux stands naturally for Real-Time Linux.

40 Note that e.g. the size of a compressed, full-sized 2.2.x kernel source tree is about 14.5 MB.



~ 20 ~

A number of implementations have proven41 RT Linux stable and robust in practice. However,one should bear in mind that, according to the reviewed related documentation, RT Linux is not afully POSIX.1c-compliant real-time operating system yet [2].

66..33.. FFIILLEESSYYSSTTEEMMSS AANNDD DDIIRREECCTTOORRYY SSTTRRUUCCTTUURREE

As is widely known, the usual access media such as disks are divided into one or more partitions,containing filesystem(s). A typeless partition is in Linux commonly used as a swap partition,making virtual memory possible.

The file system types supported by Linux include ext2fs, the Linux native type, along with anumber of other types, used for compatibility reasons. These filesystems types include theDOS/Windows compatible UMSDOS, FAT and VFAT, as well as minix, SCO Unix, etc. EvenQNX and MacOS types are supported, although only experimentally at the moment.

Each filesystem consists of a directory tree. The Linux root tree, as with most of UNIX systems,should normally contain these directories:

Essential binaries (utilities) /bin

Basic superuser-level utilities /sbin

Where most configuration files are stored /etc

Standard shared libraries /lib

All Linux devices, accessed through corresponding nodes /dev

Where the programs usually store their variable data, such as timestamps, etc. /var

A temporary catalog frequently used by running programs /tmp

Additional applications (such as X and its libraries). /usr

A virtual filesystem mounted by kernel, providing access to the process structure /proc

66..44.. BBOOOOTT AANNDD CCOOMMPPRREESSSSIIOONN

The boot process in Linux can be modified for radically different behavior of the embeddedsystem. That is why we will examine it here, as well as describe some of the alternatives. Also,since this process is mostly altered in combination with introducing some type of compression,we will subsequently examine some of these techniques.

66..44..11.. BBoooottiinngg tthhee ssyysstteemmDuring the Linux boot sequence, the system goes through the following steps [1], [3],[8],[10], [12],[13],[19], [29],[45][49]:

41 For instance, a computer running RT Linux has been successfully sent by NASA into the eye of the hurricane Georges togather data.



~ 21 ~

1. Load the kernel, which is accomplished as follows:

The (re)started system executes the BIOS code, that in turn loads the operating system codefrom the following kinds of storage:

1.1. EPROM; This alternative, as addressed in [3] calls for altering the kernel sourceand a supplementary driver so that the EPROM can be accessed as a genericblock device. Naturally, the EPROM is read-only, meaning that its contentscannot be altered at run-time in any way. M-systems DiskOnChip42 behaves in asimilar way, but is writeable. For both, BIOS boot support is needed43.

1.2. Generic block device, such as floppy, hard disk, IDE flash disk or similar, assumingthere is at least one partition dedicated to Linux. Here the MBR (Master BootRecord) is read, where one of the following can be stored:

1.2.1. The beginning of the kernel, that is subsequently loaded and to which thecontrol is then handed over.

1.2.2. A boot loader, e.g. the free LILO normally accompanying Linux, which inturn loads the kernel. LILO can also pass arguments to the kernel andprovide for additional selections.

1.2.3. DOS/Windows boot loader, starting DOS/Windows first. Then loadlin44 canbe used to boot the Linux kernel from this environment. In such a case,the kernel is loaded from an UMSDOS45-partition, which evenDOS/Windows can freely access.

Either way, when the kernel is loaded it automatically initializes basic system devices.Now the system can proceed to the succeeding step:

2. Mount the root filesystem (/). This can be done either through a local storage device or via thenetwork with e.g. NFS46. Moreover, the filesystem to be mounted can be stored in twoways, as:

2.1. A live filesystem, uncompressed and accessible, and thus directly mountable.Naturally, separate files within the filesystem can still be kept compressed, butthe filesystem itself is not unified and compressed.

2.2. A compressed image of the filesystem. In this case, a valid live filesystem is firstcreated by initializing a RAM-disk in which the compressed filesystem is thenuncompressed and stored. This process can be done by consecutively creatingtwo RAM-disks47, the first one mounting an image usually used in order to runlinuxrc , check system environment and allow loading of additional modules,

42 Otherwise, DOC could be functionally listed in 1.2.

43 Naturally, boot support is needed for other devices, but is much more common in PC-style hardware. This alternative isfrequently utilized in PC104 and custom boards.

44 Another free boot loader commonly used for this specific purpose.

45 A fully DOS-compatible filesystem type in Linux. Of course, a partition originally created by DOS is seen as UMSDOSfrom Linux, so the filesystems are compatible in both ways..

46 Network File System, common for all UNIX-like operating systems.

47 This feature is called initrd.



~ 22 ~

prior to mounting the second RAM-disk that is to contain the final rootfilesystem.

3. init is executed, checking its configuration files, in turn loading additional modules,starting network services etc., along with executing the next step.

4. Mount additional filesystems (i.e. /usr, /tmp etc.) and a swap partition, if needed. Naturally,the root tree could contain all sub-trees and no other filesystems need to be mounted.Accordingly, a swap partition is not a necessity either, provided that the system hasenough RAM for running all applications.

It is essential to note that the modules crucial for, and needed throughout the steps 1-3 have to becompiled into the kernel, unless initrd is used. This includes support for filesystem types, binaryformats and media storage/access devices (i.e. disk, RAM-disk, initrd support and network carddrivers) utilized.

66..44..22.. CCoommpprreessssiioonnAs we saw in the previous section, compression/decompression can be introduced already in theinitial boot process, but usually at cost of creating RAM-disk(s) where the uncompressedfilesystem(s) are then stored. Thus, it is important to note that the filesystems stored in the bootdevice/media, are used only during the boot itself and are not read afterwards, during run-time.The commonly used tool gzip , the well known GNU zip utility, is based on Lempel-Ziv LZ77coding and provides nine grades of compression versus speed.

66..44..22..11.. GGzzeexxee aanndd bbzzeexxee

Moreover, beside compressed filesystems one can use another very simple approach that adds alot of functionality: one can compress specific files alone, within an uncompressed and thusreadable filesystem, and have them uncompress automatically when executed. A very lucid anduseful (especially on embedded systems) shell script can be used for this purpose, gzexe. Uponrunning it on an executable binary, e.g. foo , it produces two files, foo and foo~. foo~ is the oldbinary, while foo is a newly created file consisting of a shell script in the header with acompressed binary tail. Now, when foo is executed it essentially does the following:

i. It decompresses the binary tail using g(un)zip

ii. Thereby creating an executable (the original foo ) in a temporary catalog.

iii. The binary is then executed from there.

iv. After it executes, it is automatically48 deleted, freeing the temporary space.

Similarly, bzexe49 has been made based on gzexe, but using bzip2 50 instead of gzip . bzip2 isan alternative compression tool using Burrows-Wheeler block-sorting algorithm and Hoffmancoding. Importantly, the compression grade is chosen through picking one of nine block sizesused at compression time, ranging from 100,000 to 900,000 bytes, the latter being the default.

48 The deletion is delayed, thus executed after a given amount of time.

49 bzexe has been derived from gzexe by Sebastien Huet ([email protected]), also the author of LEM, based uponan idea by Guillaume Schmid ([email protected])

50 More information can be obtained at http://sourceware.cygnus.com/bzip2/index.html



~ 23 ~

Thus, the bzip2 memory usage at both compression and decompression stage is also directlyadjusted when compressing, according to the following formulas [4]:

At compression: memory used = 400k + (7 x the chosen block size)

At decompression: memory used = 100k + (4 x the chosen block size), or if s flag is used:memory used = 100k + (2.5 x the chosen block size)

The s flag can be used at the decompression stage in order to reduce the memory usage, but thetime needed for it is then doubled. In conclusion, the compression and decompression speed isvirtually unaffected [4] at any of the stages by the block size, but as we can see the memory usageis. Also, the block size doesnt radically change the compression ratio, since most of compressioncomes from the first two or three hundred k of block size. All of the above issues can be deemed asespecially important when analyzing run-time performance and resource requirements on aconstrained system.

66..44..22..22.. ZZlliibbcc

Another compression alternative is called zlibc [57]. It takes advantage of the fact that programsnormally call the function that in turn makes system calls, and thus system calls are not done in adirect manner. This way, zlibc can intercept e.g. open() calls, and pipe the requested file,previously compressed, through g(un)zip51 , as the descriptor is returned back to the caller. Inorder to do so, the dynamic linker is redirected to use uncompress.o , and not the standardshared library. However, the use is restricted to compressing data files of executables, whichmeans that this method cannot be used for compressing executables and their shared libraries52.Also, compressing statically linked executables data files is not possible, while there is no gainfor files smaller than 1024 bytes53. In addition, it only works when files are to be read, so theycannot be automatically updated.

66..44..22..33.. EE22ccoommpprr

A third alternative, allowing live filesystems containing compressed files is currently beingdeveloped54 e2compr. Kernel patch needs to be applied, as well as a set of patched ext2fs tools55.Essentially, it conducts compression and decompression of all files marked for this feature, on thefly, in a seamless manner. However, although it apparently isnt stable yet, it could be a goodsolution when using a fully writeable filesystem, with lots of irregular reads and writes56.E2compress could thus be deemed an appropriate solution for systems where the user is giventhe freedom to make changes (add/upgrade) in the software, i.e. desktop systems. However, thisis not as usual nor welcome in most embedded systems.

51 In addition, djvu(decode) can be used. Support for bzip2 is under way.

52 The term shared library can naturally be defined as an executable as well, but we are making a clear distinction in thistext for readability reasons.

53 File system blocks are usually 1024 bytes large, so compressing such files wouldnt free any of them.

54 E2compress was not tested due to time constraints and reasons mentioned further in the text.

55 The main ext2fs tools, including e.g. e2fsck wouldnt work.

56 Also, it is said to add an overhead in the 300 KB order of value to the kernels unswappable amount of RAM usage.



~ 24 ~

66..44..22..44.. CCoonncclluussiioonn

These three (presented) alternatives use compression in different manners, and are utilizable fordifferent purposes in embedded applications. bzexe/gzexe can be used effectively only onexecutables57, while zlibc can only compress data files. E2compr could be used for all types, but isunfortunately still quite unstable.

66..55.. RRUUNN TTIIMMEE IISSSSUUEESS

The properties of an embedded Linux system involve a number of limitations that need to beaddressed. Here we will list the most important ones.

66..55..11.. WWhhyy aanndd wwhheenn rreeaadd--oonnllyy??Most of the flash storage solutions have a limited number of write cycles, usually between 100and 400 thousand although recent developments have led to devices with warranties coveringmillions of write cycles. Nevertheless, they are still limited and an analysis in write-intensiveapplications is certainly justified. Moreover, a power failure during a write cycle can, as in thecase of hard disks, damage the file system, but corrupting potentially even larger blocks. Thus,read only file systems are used as much as possible when flash is used. Obviously, EPROMcannot be written at run-time at all. Let us briefly consider what this means in Linux.

As most of the root file system need not be modified at the run-time, one can divide it intosmaller file systems, categorizing them by the time point at which they are needed, and whetherthey are modifiable. Thus, separate subsystems can be stored in various ways, compressed or not,and mounted subsequently. This way, we can either mount separate filesystems read-onlydirectly or protect the flash content by uncompressing them into a RAM-disk that in the event ofpower loss occurrences only gets deleted, not affecting the original content. At the cost of theRAM, we also get ultra fast file access.

In addition, we can use a RAM disk as a kind of a buffer for infrequently modified data, whichneeds to be saved between sessions.

As a consequence of flash usage, we should not use e.g. swap space on it, even if there wasenough room. Frequent swapping would wear the flash out rather quickly. We will get back tothis subject in the System design section on page 41 with problems and potential solutions specificfor our application, but let as meanwhile take a look at what needs to be writeable.

66..55..22.. MMooddiiffiieedd ffiillee ssyysstteemmssLinux writes frequently to the following filesystems [49]:

/var This directory is intended for variable data to start with, so writing to it is veryfrequent in ordinary Linux systems. Thus, it has to be mounted read-write, preferablyplaced in a RAM disk. Additionally, the processes writing to it (primarily a numberof daemons) can be disabled if necessary.

/tmp Many executables use this directory for storing their temporary data. The alternatives

57 Not on shared libraries.



~ 25 ~

are the same as in the previous case.

/dev This directory has to be available at boot time. Some device nodes stored here alsohave to be writeable, such as /dev/tty* written to at logon. In addition, devices likewriteable disks/disk partitions and RAM-disks can be written to this way. If any ofthese are needed, a usually better solution is linking them symbolically to a writeablepartition where these files are really stored.

/etc Commonly, only two of the files found in this directory are written to on occasionalbasis. One is /etc/mtab , modified every time a filesystem is mounted/unmounted.However, one can use n flag, and link it symbolically to /proc/mounts to avoidthis problem. The other file is /etc/HOSTNAME, written to in some cases followingDHCP58. A symbolic link to a writeable file is usually an appropriate solution evenhere.

58 Older DHCP daemons write to it instead of /var .



~ 26 ~

77.. WWWWWW BBRROOWWSSEERR OOFF CCHHOOIICCEE


For a single-function device used primarily for surfing on the web, like WebPAD, the browser isthe single most important component in terms of providing the functionality a user expects fromsuch a device. The compliance with known and widely used standards and technologies is thuscrucial. At the same time, the WebPAD concept is sensitive to the size of the embedded software,in terms of the constrained storage space and general system performance.

In this chapter, we will describe some of the browsers available for Linux and discuss theirpotential implementation on the WebPAD.

77..22.. AAVVAAIILLAABBLLEE LLIINNUUXX BBRROOWWSSEERRSS

There are a vast number of network applications available for Linux, since the Internet actuallystarted its real breakthrough with UNIX. This in turn means that there are several WWWbrowsers ported from UNIX, or being developed in parallel with their UNIX counterparts.Additionally, Linux provides an extensive number of viewers/players for virtually all knownformats and these can be used in conjunction with the browser.

However, most of these browsers were discontinued and/or do not meet the current standards.Some attempts are still at a rather early stage of development with a very slow development paceand goals set in a, commercially seen, odd fashion, and therefore they will not be discussedfurther.

Naturally, Netscape browsers are the most natural solution. They have been used on UNIX sincetheir first release, and are the most obvious solution because of their quality and stability. Let ustake a look at the design features of different versions of Netscape browsers.

77..22..11.. NNaavviiggaattoorr 22..0022Navigator 2.02 is one of the oldest Netscape legacy browsers. However, due to its robustness andrelatively extensive set of features, it is a viable alternative in resource-critical embedded systemsthat do not need compliance with latest standards. Such systems can include e.g. embeddedindustrial applications based on the WebPAD-like hardware architecture, not primarily used asclassic consumer products59.

Although it is four years old, it nevertheless includes some features that are not found in many ofits newer, non-Netscape counterparts. It is namely the first Java-enabled browser in the Netscapeseries, and it also introduced JavaScript, frames and client-side image mapping. This browsersuite also includes a fully usable MIME-compliant e-mail and news client, allowing offlinereading, as well as a bookmark maintenance facility. Yet, the browser supports just Secure

59 Although one might think that this series of browsers is forgotten, questions regarding implementation of this kind ofbrowsers are frequently posed on mailing lists and other on-line resources on embedded Linux.



~ 27 ~

Sockets Layer (SSL) 2.0. Its Gold version even incorporates a quite simple, basic HTML editor,increasing the binaries size just slightly.

Version 2.02 is only obtainable precompiled, since Netscape unfortunately never made the sourcecode of their older browsers publicly available. On Linux, the binaries are in a.out format, whichas we already mentioned, implies that the operating system must have appropriate support for it.The shared libraries needed are modest in size. The ideal size-optimized system incorporatingNavigator 2.02 would be a libc4 system with the older a.out X Windows, where applicable.

However, the browser itself is about 3.8 MB in the uncompressed state, and the Java classes files,already compressed60, take up 748 KB. It does not support plug-ins, although it can be used incombination with external viewers and players.

77..22..22.. NNaavviiggaattoorr 33..0044Navigator 3.04, released in 1997, is generally a much better alternative. In addition, it markedNetscapes transition from a.out to ELF binary format and libc5, with respect to the Linuxversion. The browser includes essentially the same set of features as its predecessor, but it isfurther refined, upgrading to: JavaScript 1.1, enhanced security (SSL 3.0 and digital certificates)and adding full support for plug-ins, Linux inclusive. Its e-mail and news clients are now RichHTML capable, and the tags supported include those of them proposed by the HTML 3.2specification.

This browser can most certainly be denoted as a die-hard alternative. The executable is sized at4.3 MB and the Java classes originally compressed at 713 KB. The ideally size-optimized systemis obviously libc5/ELF-based, running libc5 version of X.

77..22..33.. NNaavviiggaattoorr 44..xxxxWith its 4.xx suite, Netscape entirely moved on to the modular approach, slashing the suite inseveral components. However, the browser-only package still includes a monolithic binaryexecutable. Jar-files are now extended with e.g. localization and different platform support. Newfeatures are added, such as push/channels, dynamic fonts61 and Cascading Style Sheets (CCS), asJavaScript has been updated to 1.3.

It also marked Netscapes introduction of glibc2-based browsers. Version 4.08 is the last browseravailable only for libc5, as 4.5x and 4.6x are also available for glibc2.

The browser executable is now 7.1 MB in size62, and the added default plug-in63 is 788 KB.However, it doesnt necessarily have to be included64. Java classes have been extended as well,and have, with support for e.g. Lightweight Directory Access Control (LDAP) and the ratherextensive internationalization support grown to 4.3 MB. However, the Java support should beadequate even if these files reduced to about 3 MB.

60 Here, they are stored in zip format, and not in gz (GNU zip) as in v3.04 or jar (Java Archive) as in later versions.Additionally, jar seems to really be gz in Linux.

61 According to Netscape Technical Support, only on 4.61.

62 Navigator 4.61 Stand-alone, glibc2 variant. Libc5 versions, particularly 4.0x, are somewhat smaller.

63 libnullplugin.so , handles unknown MIME types and prompts the user for download of the proper plug-in.

64 It can be skipped. Alternatively, one could create a new one or apparently even modify the Mozilla default plug-in.



~ 28 ~

77..22..44.. MMoozziillllaa aanndd NNaavviiggaattoorr 55 ttoo--bbeeIn 1998 Netscape released the source code of their early version 5 browser suite65, under NetscapePublic License (NPL), initiating the Mozilla project [28]. Mozilla, consisting of an enormousnumber of developers tightly cooperating with Netscape, took on the further development of thecode.

However, it is important to note that the released code doesnt include those parts that werecopyrighted by third parties and legally non-distributable modules. Hence, the source code ofMessenger, Netscapes e-mail client, as well as Suns Java VM and SSL module are not included inMozilla. Java can however be built in through OJI (Open Java VM Interface), allowing for OJIJVM plug-ins. One such, GNU Japhar is currently available.

The fast and modularized browser engine used, supporting HTML 4.0, CCS 1 and 2 as well asJavaScript 1.5. XML 1.0 and DOM, has been the cause of much confusion and manymisconceptions. Netscape calls it Gecko66, while it is also called NGLayout, especially in theMozilla community67. The latter however also uses the names XPFE and XPToolkit for theengines user interface toolkit, while strictly the layout engine itself is denoted as NGLayout68.Netscape intends to package Mozilla in its future releases of the 5-series, as will everyone else.

Linux is one of the reference platforms used and one of the main targets. The development isbased solely on glibc2.1 and is planned to tightly follow further releases of glibc. The toolkit usedis GNU Toolkit (GTK)69. The browser package is very modularized, consisting of a smallexecutable, apprunner/viewer, loading numerous smaller libraries which add features. Naturally,the complete source tree is available.

However, although the set of features is very extensive, it is still under heavy development. TheMozilla release, M6 (Milestone 6) which we tested, was still quite buggy and rather unstable.

77..33.. BBRROOWWSSEERR CCUUSSTTOOMMIIZZAABBIILLIITTYY

77..33..11.. RReemmoottee CCoonnttrrooll aanndd IInntteerr--AApppplliiccaattiioonn CCoommmmuunniiccaattiioonnAs noted before, the binary-distributed Netscape browsers consist of a single binary, in additionto potential plug-ins and such, as well as Java classes. The browser executable is monolithic,which makes customization more difficult. However, run-time behavior customization isachievable to a rather high extent by using Remote Control and Inter-Application Communicationthrough X events [[56].

Accordingly, a running browser instance gladly responds to Netscapes built-in Xt actions, e.g. remote option, to which a number of actions can be fed. For instance, simply executing

sh> netscape remote 'openURL (http://www.it.kth.se)'

65 It is not source of beta release, as we will see.

66 It has nothing to do with engines implemeted in 4.xx and previous Netscape releases, but is totally new.

67 And it gets worse. Netscape used to refer to it as NGT, formerly known as Raptor.

68 NGLayout is also used for rendering the GUI.

69 Also used in e.g. gnome.



~ 29 ~

at the shell command prompt70 would open this link in the existing browser window. Leaving theparentheses empty would on the other hand prompt the user to enter the URL. Beside severalother arguments that can be sent via openURL, there are additional actions utilizing argumentsthis way, such as openFile , saveAs, addBookmark and mailto 71. The vast number of otherimplemented X resources72, allows further customization although these do not take argumentsas described above. The window that one wants to be controled remotely is identified by idoption. Similarly, one can control whether and ho

embedded linux for purposes of internet appliances

Documents

embedded linux

benefits of linux

system architecture

system premises

new internet appliances

x window system

system size reduction

operat ing system