Download - LCA14: LCA14-416: Big Endian - where are we?
![Page 1: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/1.jpg)
Thu 6 March, 4:10pm, Victor Kamensky
LCA14-416: Big Endian - Where are we?
![Page 2: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/2.jpg)
• Mandatory read:Big and Little Endian Inside Out – Ben Herrenschmidt, LPC12
• ARM CPU architecture is endian agnostic - can do either• ARM Linux ecosystem traditionally is Little Endian• Surge of interest in ARM Big Endian support
What is Big and Little Endian?
![Page 3: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/3.jpg)
• Very big, 100M+ LOC, legacy code, Big Endian only• Runs on MIPS and PPC CPUs, looking forward to ARM
CPU adoption• Not because Big Endian is better for networking
applications
Why is Big Endian?
![Page 4: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/4.jpg)
• Last few years ARM Linux runs in Little Endian mode• XScale CPU runs in BE32 mode• Big Endian BE32 ecosystem for XScale CPUs still exists
in distros like OpenWRT• Some vendors offered very minimal V7 BE BSP support
ARM Big Endian Linux History
![Page 5: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/5.jpg)
• Build reasonable, supported, Linux Big Endian ecosystem for ARM V7 and V8 CPUs. Support as “second class citizen” is good enough
• Mixed mode support where Little Endian and Big Endian code could coexist on the same box
• Tools and methodologies that could help transition Big Endian Only code to Endian Agnostic
LNG Endian Team was Set to Do
![Page 6: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/6.jpg)
• Truly great open source effort• Effort got help from many companies and individuals• Few to mention, but not limited:
• Codethink• ARM• Linaro
• Thank you!
We Got Help … a Lot!
![Page 7: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/7.jpg)
• V7 ARM BE8 gcc support existed for a while• V8 ARM BE gcc support was verified/fixed by Linaro
toolchain group• Linaro toolchain releases for gcc 4.8 include Big Endian
variants:• V7 - armeb• V8 - aarch64_be
• Some issues may remain for Neon vector intrinsics
ARM V7 & V8 Big Endian Toolchain
![Page 8: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/8.jpg)
• Bootloader, u-boot or uefi, remains Little Endian• All typical SoCs peripheral devices are Little Endian
ARM Big Endian Kernel Context
![Page 9: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/9.jpg)
• BE8 ARM V7 core support was driven by Ben Dooks from Codethink
• LNG Endian team collaborated with Ben: testing, fixes, review
• BE8 V7 core support series appeared in 3.13
ARM V7 Big Endian Kernel Core Support
![Page 10: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/10.jpg)
• Some just worked … with very minor changes• TC2• Mvebu• Highbank
• LNG Endian team use TI Pandaboard and Samsung Arndale as reference and development platform for Big Endian work (a lot of regular driver fixes)• TI Pandaboard changes are in process of being upstreamed• Samsung Arndale changes no intention to upstream
• LNG Endian team helped, in different degrees, to enable Big Endian support on• TI Keystone• LSI Amarillo
ARM V7 Big Endian Kernel BSP
![Page 11: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/11.jpg)
• Supported through Linaro OE• Changes are in meta-linaro/meta-bigendian• Very few user-land packages are affected• Few examples:
• strace• python libffi
ARM V7 Big Endian User-land
![Page 12: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/12.jpg)
• LNG Endian team tested and fixed different kernel functional areas which could be affected by endianity change
• Just to mention few:• BE kexec/kdump - minor fixes• BE kprobes - upstream pull request posted• BE uprobes - folded into regular Linaro uprobes work• BE KVM host support - patches posted for upstream review
ARM V7 Big Endian Kernel Advanced Features
![Page 13: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/13.jpg)
• Done by ARM folks: Matthew Leach, Will Deacon• Was very nice surprise• Included in 3.13• Works/tested on fastmodels and real V8 h/w
ARM V8 Big Endian Kernel Core Support
![Page 14: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/14.jpg)
• Big Endian V8 KVM host support patches posted for upstream review
• We will test other advanced features on V8 as soon as they are available on V8
ARM V8 Kernel Big Endian Advanced Features
![Page 15: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/15.jpg)
• Supported through Linaro OE• aarch64_be target support was added to
openembedded-core with help of Linaro release team• Ran into few minor issues
ARM V8 Big Endian User-land
![Page 16: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/16.jpg)
• Java• Valgrind• Gcc neon vector intrinsics
Anything else? What did we miss?
Known Big Endian ARM Linux Features that don’t Work
![Page 17: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/17.jpg)
• “Special” Big Endian user-land process on top of Little Endian Kernel along side with regular Little Endian processes
• Byte-swaps on system call and ioctl level in kernel• Cisco conducted some experiments in this area:
• converted around 30 system calls and 20 ioctls• messy, big, intrusive change• non-trivial code in futex area
• Byte-swaps before calling Little endian function:• did not actually try• but seems even more harder• i.e how to debug process that may have Big Endian and Little Endian
code coexisting
Mixed Mode: Big and Little Endian
![Page 18: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/18.jpg)
• Done by Marc Zyngier• Looks nice and clean• See Crossing the endianness bridge - Marc Zyngier KVM Forum '13• Marc showed Big Endian guest running on top of Little
Endian KVM host• Require changes in emulator - patches for kvmtool by
Marc are available• Guest need to treat virtio config as LE; not committed
yet; may need to wait for virtio-2 support• Once Big Endian KVM support appeared it could run
Little Endian guests (tested on V7 and V8)• If we have time will make a demo of it
Mixed Mode: through KVM
![Page 19: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/19.jpg)
• Cisco runs big project in this area, attempting to convert 100M+ Big Endian only LOC into Endian agnostic code
• Effort is much larger compared to anything LNG Endian team did
• Approach outline:• sparse with a lot of changes used to annotated code and detect
endian issues• sparse tool was changed to do source to source transformation• python plugin added that allows write external rules how to transform
code• Considered very risky and big, may take long time• Cisco most likely will be able to share and present
details about the effort by next Linaro Connect (tentative agreement reached)
Big Endian to Endian Agnostic Tools
![Page 20: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/20.jpg)
• ARM V7 and V8 Big Endian basic ecosystem looks good• Mixed mode support should be addressed through KVM
virtualization• Big Endian porting mainly will be based on Cisco internal
project experience
Summary
![Page 21: LCA14: LCA14-416: Big Endian - where are we?](https://reader033.vdocuments.net/reader033/viewer/2022052619/5561eef2d8b42ab1068b5649/html5/thumbnails/21.jpg)
More about Linaro Connect: http://connect.linaro.orgMore about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/Linaro members: www.linaro.org/members