big data technology - cloud computing

90
Prof. 潘潘潘 Pan, Ren-Hao 潘潘潘潘•潘潘潘潘潘潘潘潘潘潘 Big Data Technology – Could Computing

Upload: ren-hao-pan-pan

Post on 03-Aug-2015

861 views

Category:

Education


8 download

TRANSCRIPT

1. Prof. Pan, Ren-Hao 2. The Obligatory Timeline Slide (Mike Culver @ AWS) COBOL, Edsel Amazon.com Darkness Web as a Platform Web Services, Resources Eliminated Web Awareness InternetARPANET Dot-Com Bubble Web 2.0 Web Scale Computing 3. ? 4. (Cloud Computing) 5. (Cloud Computing) (Cloud Computing) ()Cloud Computing Technologies (massively) (scalable) (elastic) 6. Benefits Cost & management Economies of scale, out-sourced resource management Reduced Time to deployment Ease of assembly, works out of the box Scaling On demand provisioning, co-locate data and compute Reliability Massive, redundant, shared resources Sustainability Hardware not owned 7. Types of Cloud Computing Public Cloud: Computing infrastructure is hosted at the vendors premises. Private Cloud: Computing architecture is dedicated to the customer and is not shared with other organizations. Hybrid Cloud: Organizations host some critical, secure applications in private clouds. Not critical applications are hosted in the public cloud Cloud bursting: the organization uses its own infrastructure for normal usage, but cloud is used for peak loads. 8. (Cloud Computing) (Distributed Computing) (Cluster Computing) (Grid Computing) (Cloud Computing) 9. (Cloud Computing) (Cluster Computing) Networked in a tightly-coupled fashion The same subnet of the same domain Nodes are homogeneous The same hardware The same software Grid and Cluster Computing: Options for Improving Windows Application Performance : http://www.cs.mtu.edu/beowulf/ 10. (Cloud Computing) (Grid Computing) Loosely-coupled Across domains or subnets Nodes can be heterogeneous Diverse hardware and software configurations Grid and Cluster Computing: Options for Improving Windows Application Performance : http://tech4globe.com/what-is-grid-computing.html 11. (Cloud Computing) (Grid Computing) SETI@HOME Folding@Home 2000.10.01 12. (Cloud Computing) : (Market Intelligence & Consulting Institute, MIC 13. (Cloud Computing) vs. (Cloud Service) 14. (Cloud Computing) ()Cloud Computing Technologies (Cloud Services) 15. (Cloud Services) (Key Cloud Services Attributes) 1. Offsite, provided by third-party provider 2. Accessed via the Internet 3. Minimal/no IT skills to implement 4. Provisioning : self-service requesting, near real- time deployment, dynamic & fine-grained scaling. IDC, 2008 16. (Cloud Services) (Key Cloud Services Attributes) (cont.) 5. Pricing : fine-grained & usage-based pricing capability 6. User Interface : browser & successors 7. System Interface : Web services APIs 8. Shared resources/common versions : customize around the shared services IDC, International Data Corporation, 2008 17. 18. - Google App Engine 19. Basic LAMP Google Developer Day Slide 20. Dedicated Database Google Developer Day Slide 21. Multiple Web Servers Google Developer Day Slide 22. Load Balance: DNS Round Robin Google Developer Day Slide 23. Load Balance: DNS Round Robin Google Developer Day Slide 24. Load Balance: DNS Round Robin Google Developer Day Slide 25. Reverse Proxy Google Developer Day Slide 26. Reverse Proxy Google Developer Day Slide 27. Master-Slave Database Google Developer Day Slide 28. Master-Slave Database Google Developer Day Slide 29. Master-Slave Database Google Developer Day Slide 30. Partitioned Database Google Developer Day Slide 31. Cloud Solution : Google App Engine Google Developer Day Slide 32. Cloud Solution : Hadoop File System + HBase Google Developer Day Slide 33. 2011 ABC News - Japan Earthquake: before and after - Link Google - Link - Link 34. 35. wikipedia 36. https://www.ibm.com/developerworks/mydeveloperworks/blogs/sbose/entry/gathering_clouds_of_xaas?lan g=en 37. (Software as a Service, SaaS) Internet(Software on Demand) Google , Microsoft, Salesforce.com 38. (platform as a service, PaaS) PaaS Google App Engine, Microsoft Azure, 39. Google App Engine vs. Hadoop 40. (Infrastructure as a service, IaaS) IT IT IaaS IT EC2 (Amazon Elastic Compute Cloud) Google Storage hicloud Hadoop 41. (Infrastructure as a service, IaaS) /(Virtualization) 42. Infrastructure as a Service (IaaS) 43. Classification of Cloud Computing based on Service Provided Infrastructure as a service (IaaS) Offering hardware related services using the principles of cloud computing. These could include storage services (database or disk storage) or virtual servers. Amazon EC2, Amazon S3, Rackspace Cloud Servers and Flexiscale. Platform as a Service (PaaS) Offering a development platform on the cloud. Googles Application Engine, Microsofts Azure, Salesforce.com, s force.com . Software as a service (SaaS) Including a complete software offering on the cloud. Users can access a software application hosted by the cloud vendor on pay- per-use basis. This is a well-established sector. Salesforce.coms offering in the online Customer Relationship Management (CRM) space, Googles gmail and Microsofts hotmail, Google docs. 44. More Refined Categorization Storage-as-a-service Database-as-a-service Information-as-a-service Process-as-a-service Application-as-a-service Platform-as-a-service Integration-as-a-service Security-as-a-service Management/ Governance-as-a-service Testing-as-a-service Infrastructure-as-a-service InfoWorld Cloud Computing Deep Dive 45. Virtualization 46. Utility Computing Software as a Service Service Oriented Architecture Software + Services Software as Self Service Platform as a Service Matias Woloski, Cloud Computing Taxonomy Map 47. Amazon Web Services (AWS) Elastic Compute Cloud EC2 (IaaS) Simple Storage Service S3 (IaaS) Elastic Block Storage EBS (IaaS) SimpleDB (SDB) (PaaS) Simple Queue Service SQS (PaaS) CloudFront (S3 based Content Delivery Network PaaS) Consistent AWS Web Services API Link : http://aws.amazon.com/products/?nc2=h_ls 48. What does Azure platform offer to developers? 49. 50. Coca-Cola village and Facebook Promarket () RFID Facebook Video Link Zara - Inditex RFID Project Video Link 51. 52. (99 .4 ) 5 240 1,000 127 () 1,000 5 1 53. 54. - 1 C4 55. 1 C4 (Cloud) (Commerce) (Connectivity) (Client) 56. 2 /G-Cloud (G2G) IT (G2B,G2B2B) //- e (G2C) 57. 2 /G-Cloud 58. 3 59. - /99 ~ 103 / 16.5 5i 60. 61. - 5i Internet Interactive Integrate Incentive (web2.0 ) Innovative http://edumeaning.51donate.com/ 62. 63. 64. 65. - 66. EuroCloud Issues : strict privacy laws that place rigid limits on the movement of information beyond the borders the 27-country European Union Cloud Computing Hits Snag in Europe, 2010 67. IaaS PaaS SaaS hicloud /TCloud Elaster Acer eDC /ITIL /for Enterprise WebStorage/for ASUS Product G-Cloud - -TCloud CAP(Cloud App. Platform) - - - - -/ SaaS CRM - - - XDNA,PM -SOA-ERP IT ( . .) ,2010 68. 69. (Cloud Security Alliance, CSA) Google 2010.03 - Top Threats to Cloud Computing V1.0 70. CSA 1. (Abuse and Nefarious Use of Cloud Computing) IaaS , PaaS 2. APIs (Insecure Interface and APIs) IaaS , PaaS, SaaS 3. (Malicious Insiders) IaaS , PaaS, SaaS ISMS 71. CSA 4. (Shared Technology Issues) IaaS (ex: RAM), (ex:VLAN), 5. (Data Loss or Leakage) IaaS , PaaS, SaaS ISMS, (ex: , ),, , AAA ( ) 6. (Account or Service Hijacking) IaaS , PaaS, SaaS ISMS, (ex:), Token 72. CSA 7. (Unknown Risk Profile) IaaS , PaaS, SaaS , , 73. WPA Cracker - A Cloud cracking service WPA-PSK protected wireless networks ZIP cracking 284 million word dictionary 400CPU cluster 74. Privacy Protection IaaS security PIaaS security SaaS security 100 75. (Privacy Protection) Web 76. IaaS security (virtual machine independency) block level 77. PaaS security AAA (Authentication, Authorization & Accounting) 78. SaaS security Web App XML, RIA Web DB Behavior based AAA (Authentication, Authorization & Accounting) 79. 80. CIA Confidentiality () Integrity () Availability () Cross-layer security solutions 81. 82. International Data Corporation (IDC) - Cloud Computing 2010 - An IDC Update Cloud User Surveys Benefits Challenges Adoption Areas Vendor Requirements 83. Cloud User Surveys - Benefits 54.0% 63.9% 64.6% 67.0% 68.5% 75.3% 77.7% 77.9% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Seems like the way of the future Sharing systems with partners simpler Always offers latest functionality Requires less in-house IT staff, costs Encourages standard systems Monthly payments Easy/fast to deploy to end-users Pay only for what you use Cloud Computing 2010 . An IDC Update 84. Cloud User Surveys - Challenges 76.0% 76.8% 79.8% 80.2% 81.0% 82.9% 83.3% 87.5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Not enough ability to customize Hard to integrate with in-house IT Bringing back in-house may be difficult Lack of interoperability standards On-demand paymt model may cost more Performance Availability Security Cloud Computing 2010 . An IDC Update 85. Cloud User Surveys Adoption Areas 48.6% 49.1% 49.8% 50.6% 51.3% 52.9% 54.8% 55.1% 55.6% 59.4% 66.9% 67.3% 0% 10% 20% 30% 40% 50% 60% 70% 80% IT/Information Security Application dev/test/deploy platform Business Intelligence/Analytics Server capacity on demand IT Management software Storage capacity on demand Data/Content Distribution services Personal productivity apps Business apps (CRM, HR, ERP) Data Back-up or Archive services Web applications/Web serving Collaboration applications Cloud Computing 2010 . An IDC Update 86. Cloud User Surveys Vendor Requirements 72.9% 78.3% 79.2% 81.0% 82.1% 84.5% 86.0% 87.8% 88.6% 91.6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Have local presence, can come to my offices Are a technology and business model innovator Offer both on-premise and public cloud services Support many of my IT needs Allow managing on-premise & cloud together Understand my business and industry Provide a complete solution Option to move 'cloud' offerings back on premise Offer Service Level Agreements (SLAs) Offer competitive pricing Cloud Computing 2010 . An IDC Update 87. Rodrigo Flores (CTO, newScale, Inc.) The Competitive Threat of Public Clouds Give up the fight IT => IT/ Define your Model T Ford / Think like an ATM (Self-Service) Model T 1913 Assembly line 121 88. Thanks E-Mail : [email protected]