developmental aids whitepaper 9-1-2017 › media › files › white_paper › ... · apache sentry...
TRANSCRIPT
DEVELOPMENTAL AIDSHOW THIRD PARTY SOFTWARES ARE MAKING LIFE EASIER IN THE DEVELOPMENT WORLD
IntroductionAgile projects are executed at break-neck speed nowadays. Finishing the
project in time and within budget without compromising on quality has
become a bigger challenge than in the past. This is compounded by issues
like design complexity and the difficulties of confining development effort to
time-boxed sprints. This is where the advent of cool aids in the form of a host
of third party software has proved to be a big boon for project teams. So let
us take a look at some of the prominent aids.
AWESOME
DEVELOPMENTAL
AIDS
www.sayonetech.com 02
Developmental Aids
Crashlytics
Google Analytics
Deeplinking
Papertrail
Apache Bench
Logentries
New Relic
Index
04
05
07
08
09
10
11
www.sayonetech.com 03
Developmental Aids
Apache Sentry
Apache JMeter
Amazon Web Services
Mailgun
SendGrid
Campaign Tracker
Summary
12
13
15
17
18
19
20
CrashlyticsCrashlytics is a popular software development kit (SDK) owned by Twitter. It helps iOS and Android developers to pinpoint issues causing their app’s
instabilities. With Crashlytics’ service, which includes its own online dashboard, crash logs are distilled down; highlighting the real issues, detailing the device’s
state at the time of crash (software version, orientation, model, etc.) and it even shows developers the exact line of code that the app crashed on. Crashlytics is
the most powerful, yet lightest weight crash reporting solution providing reliable performance SDK on both iOS and Android, Crashlytics provides deep and
actionable insights. Over 2 billion active devices have Crashlytics installed in them. The company serves thousands of companies, including Yelp, Kayak,
PayPal, Walmart, Groupon, Waze, Blackboard, Path, OpenTable, Domino’s, Expedia and Square.
www.sayonetech.com 04
Developmental Aids
Google AnalyticsGoogle Analytics is a freemium web analytics service offered by Google that tracks and reports website traffic. Google Analytics shows high-level,
dashboard-type data for the casual user and more in-depth data further into the report set. Google Analytics analysis can identify poorly performing pages
with techniques such as funnel visualization, where visitors came from (referrers), how long they stayed and their geographical position. It also provides
more advanced features, including custom visitor segmentation.
Google Analytics e-commerce reporting can track sales activity and performance. The e-commerce report shows a site's transactions, revenue, and many
other commerce-related metrics. A user can have 100 site profiles. Each profile generally corresponds to one website. It is limited to sites which has traffic
of fewer than 5 million page views per month unless the site is linked to an AdWords campaign.
Google Analytics includes Google Website Optimizer, rebranded as Google Analytics Content Experiments.
Google Analytics Cohort analysis feature helps to understand the behavior of component groups of users apart from your user population. It is very much
beneficial to marketers and analysts for successful implementation of Marketing Strategy.
www.sayonetech.com 05
Developmental Aids
Developmental Aids
www.sayonetech.com 06
Google Analytics is implemented with page tags, called the Google Analytics Tracking Code, which is a snippet of JavaScript code that the website owner
adds to every page of the website. The tracking code runs in the client browser when the client browses the page (if JavaScript is enabled in the browser)
and collects visitor data and sends it to a Google data collection server as part of a request for a web beacon.
The tracking code loads a larger JavaScript file from the Google web server and then sets variables with the user's account number. The larger file
(currently known as ga.js) is typically 18 KB. The file does not usually have to be loaded, however, due to browser caching. Assuming caching is enabled in
the browser, it downloads ga.js only once at the start of the visit. Furthermore, as all websites that implement Google Analytics with the ga.js code use the
same master file from Google, a browser that has previously visited any other website running Google Analytics will already have the file cached on their
machine.
In addition to transmitting information to a Google server, the tracking code sets a first party cookie (If cookies are enabled in the browser) on each visitor's
computer. This cookie stores anonymous information, called the Client Id. Before the launch of Universal Analytics, there were several cookies storing
information such as whether the visitor is new of had been to the site before, timestamp of the current visit, and the referrer site or campaign that directed
the visitor to the page (e.g., search engine, keywords, banner, or email).
Google Analytics
DeeplinkingIn the context of the World Wide Web, deep linking is the use of a hyperlink that links to a specific, generally searchable or indexed, piece of web content on a
website (e.g., http://www.sayonetech.com/services/web-applications/), rather than the website's home page (e.g., http://www.sayonetech.com/). Branch.io
refers to the open source mobile linking service from Branch Metrics. The Branch Metrics SDK enables developers to integrate their apps with deep linking,
contextual deep linking, organic, paid and referral attribution, and app-to-app communication. Use the branch.io tag for questions relating to Branch's Web,
iOS, and Android SDKs, Branch Links and registering Webhooks.
www.sayonetech.com 07
Developmental Aids
PapertrailPapertrail is a cloud-based, log-management service. Machines’ log files have always been important for troubleshooting, but are seeing increased interest
with the advent of big data because companies have realized they can draw business-level insights from log data, too. Papertrail is gaining popularity at a time
when organizations are embracing both cloud services and big data tools. Papertrail offers a place to store, search and analyze log files from a number of
different sources for a low monthly fee. The service can handle a variety of operating system logs, app server requests, database queries, and router logs.
Papertrail’s features include a graphical dashboard, search capabilities via command line or RESTful API, and email alerts. Papertrail is hosted on Amazon’s
EC2 infrastructure and advertises long-term storage using Amazon’s S3 storage service and advanced analytics using Amazon’s Hadoop-based Elastic
MapReduce service. Papertrail aggregates the following.
syslog
Text log files
Apache
MySQL
cloud hosting
Ruby on Rails
Windows events
Tomcat
Heroku apps
Routers & firewalls
Highlights
Instant log visibility: Time-saving log tools,
flexible system groups, team-wide
access, long-term archives, charts and
analytics exports, monitoring web hooks,
and 45-second setup.
Value from collected logs: Track down
customer problems, error messages, app
requests, slow DB queries, config
changes.
www.sayonetech.com 08
Developmental Aids
ApacheBenchApacheBench (ab) is a single-threaded command line computer program for measuring the performance of HTTP web servers. Originally designed to test the
Apache HTTP Server, it is generic enough to test any web server. The ab tool comes bundled with the standard Apache source distribution, and like the
Apache web server itself, is free, open source software and distributed under the terms of the Apache License. It is designed to give you an impression of how
your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving. Apache-
Bench will only use one operating system thread regardless of the concurrency level (specified by the -c parameter). In some cases, especially when bench-
marking high-capacity servers, a single instance of ApacheBench can itself be a bottleneck. When using ApacheBench on hardware with multiple processor
cores, additional instances of ApacheBench may be used in parallel to more fully saturate the target URL.
www.sayonetech.com 09
Developmental Aids
LogentriesLogentries is a software-as-a-service provider for log management and intelligence. Logentries collects and analyzes data found within log files in real-time
with a cloud-delivered approach. Logentries has more than 25,000 users. The Logentries technology collects and analyzes logs across software stacks using
a pre-processing layer to filter, correlate, and visualize log data. The tool uses a combination of AWS tools and open source applications to deliver the
application.
FeaturesWhether your data is in structured JSON or mysterious plain text, it’s easy to send to Logentries for immediate search.
Organize logs from servers, applications, routers, containers and more in one central location.
Search is really fast, whether one is searching keywords, key value pairs or regex patterns.
Powerful Analytics: Analytical functions such as AVERAGE(), COUNT(), GROUPBY(), min, max and percentile can be used with its intuitive query language,
LEQL
Robust Visualizations: Analyze log data with line graphs, bar charts, multi-line reports and more.
Simple export tools and APIs make it easy to share data outside of Logentries to view it using tools like Tableau, Hosted Graphite or Geckoboard.
Logentries’ unique preprocessing engine enables you to watch your logs stream directly into the Logentries UI with Live Tail and receive immediate alerts
without delay.
Catch unusual activity with live Anomaly Detection based on automated analysis of your systems’ baselines.
Publish alerts to popular tools like Slack, Hipchat, PagerDuty or any webhook for seamless integrations.
REST Alerting API
www.sayonetech.com 10
Developmental Aids
New RelicNew Relic is a software analytics technology delivered in a software-as-a-service (SaaS) model. It monitors Web and mobile applications in real-time that run
in cloud, on-premises, or hybrid environments. The name ‘New Relic’ is an anagram of the name of the founder Lew Cirne. New Relic launched an open SaaS
platform designed to allow developers to deploy 50+ plug-ins from technology partners (or build their own) to the New Relic dashboard. Plug-in technologies
include PaaS/cloud services, caching, database, Web servers and queuing.
The New Relic service has 70 partnerships around the world including include IBM Bluemix, Amazon Web Services, CloudBees, Engine Yard, Heroku, Joyent,
Rackspace Hosting, and Microsoft Azure as well as mobile application backend service providers Appcelerator, Parse, and StackMob. New Relic can be used
to monitor the app’s performance and also other things like database performance, load balancers, caching, or even business data like conversion rates. Any
kind of actionable data can be dreamt up and used for building a plug-in for the New Relic dashboard. New Relic has a fancy API and two SDKs (Ruby and
Java) for programming and a UI builder to keep the dashboard clean and the process fast.
The platform has plug-ins for several technologies like Apache, Nginx, CouchDB and Redis, Varnish and Memcache as well as less popular programs like
SendGrid. The API and SDKs are well maintained and exhaustively documented. Once a plug-in is complete, it can be used privately within your own
organization, or you can join in the free-for-all and publish it publicly to Plug-in Central, a sort of app store for New Relic plugins where others can download
and use your work.
www.sayonetech.com 11
Developmental Aids
Apache SentryApache Sentry is a granular, role-based authorization module for Hadoop. Sentry provides the ability to control and enforce precise levels of privileges on data
for authenticated users and applications on a Hadoop cluster. Sentry currently works out of the box with Apache Hive, Hive Metastore/HCatalog, Apache Solr,
Impala and HDFS. Sentry is designed to be a pluggable authorization engine for Hadoop components. It allows you to define authorization rules to validate a
user or application’s access requests for Hadoop resources. Sentry is highly modular and can support authorization for a wide variety of data models in
Hadoop. Sentry can be used to enforce various access policy rules when accessing data stored on Hadoop Distributed File System through various Hadoop
ecosystem components such as Apache Hive, Apache Pig or others.
Traditionally, user access control in Apache Hadoop has been implemented using file based permissions on HDFS. Following the UNIX permissions model,
HDFS offers all or nothing semantics allowing administrator to configure system to allow certain users or user groups read, write or perform both operations on
files. This system does not enable more fine grained permissions that allow access policies for logical parts within one file. Furthermore, this model can't be
used to restrict access to the rich set of objects in the metadata catalog that are stored outside HDFS.
Sentry was initially developed by Cloudera to allow users fine grained access to data as well as the metadata in Apache Hadoop. Sentry has been maintained
as an open source project on Cloudera’s github. All code in Sentry is open source and is publicly available under the Apache 2 license.
www.sayonetech.com 12
Developmental Aids
Apache JMeterThe Apache JMeter application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance.
Apache JMeter can be used to test performance both on static and dynamic resources (Web Services (SOAP/REST), Web dynamic languages - PHP, Java,
ASP.NET, Files, etc. -, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, group of servers,
network or object to test its strength or to analyze overall performance under different load types. It can make a graphical analysis of performance or test your
server/script/object behavior under heavy concurrent load.
Apache JMeter features include:
Ability to load and performance test many different server/protocol types:
Web - HTTP, HTTPS
SOAP / REST
FTP
Database via JDBC
LDAP
Message-oriented middleware (MOM) via JMS
Mail - SMTP(S), POP3(S) and IMAP(S)
Native commands or shell scripts
TCP
Complete portability and 100% Java purity.
Careful GUI design allows faster Test Plan building and debugging.
www.sayonetech.com 13
Developmental Aids
www.sayonetech.com 14
Developmental Aids
The Apache JMeter application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance.
Apache JMeter can be used to test performance both on static and dynamic resources (Web Services (SOAP/REST), Web dynamic languages - PHP, Java,
ASP.NET, Files, etc. -, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, group of servers,
network or object to test its strength or to analyze overall performance under different load types. It can make a graphical analysis of performance or test your
server/script/object behavior under heavy concurrent load.
Apache JMeter features include:
Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
Caching and offline analysis/replaying of test results.
Highly Extensible core:
Pluggable Samplers allow unlimited testing capabilities.
Several load statistics may be chosen with pluggable timers.
Data analysis and visualization plugins allow great extensibility as well as personalization.
Functions can be used to provide dynamic input to a test or provide data manipulation.
Scriptable Samplers (BeanShell, BSF-compatible languages and JSR223-compatible languages)
Apache JMeter
Amazon Web ServicesAmazon Web Services (AWS), a subsidiary of Amazon.com offers a suite of cloud-computing services that make up an on-demand computing platform. These
include Amazon Elastic Compute Cloud, also known as "EC2", and Amazon Simple Storage Service, also known as "S3". AWS has more than 70 services,
spanning a wide range, including compute, storage, networking, database, analytics, application services, deployment, management, mobile, developer tools
and tools for the Internet of things. Amazon markets AWS as a service to provide large computing capacity quicker and cheaper than a client company
building an actual physical server farm.
Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your
computing resources and lets you run on Amazon’s proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server
instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the
economics of computing by allowing you to pay only for capacity that you actually use. Amazon EC2 provides developers the tools to build failure resilient
applications and isolate themselves from common failure scenarios.
www.sayonetech.com 15
Developmental Aids
Compute - Amazon Elastic Beanstalk provides a PaaS service for hosting applications, equivalent services include Google App Engine or Heroku or
OpenShift for on-premises use.
Networking - Amazon Virtual Private Cloud (VPC) creates a logically isolated set of AWS resources which can be connected using a VPN connection
Content Delivery: Amazon CloudFront, a content delivery network (CDN) for distributing objects to so-called "edge locations" near the requester.
Storage - Amazon Simple Storage Service (S3) provides Web Service based storage.
Database - Amazon Redshift provides petabyte-scale data warehousing with column-based storage and multi-node compute.
AWS CodeDeploy provides automated code deployment to EC2 instances.
Management - AWS Directory Service a managed service that allows connection to AWS resources with existing on-premises Microsoft Active Directory or
to set up a new, stand-alone directory in the AWS Cloud.
Application Services: Amazon Simple Email Service (Amazon SES) is a cost-effective email service built on the reliable and scalable infrastructure that
Amazon.com developed to serve its own customer base. With Amazon SES, you can send and receive email with no required minimum commitments – you
pay as you go, and you only pay for what you use.
Analytics: Amazon Machine Learning a service that assists developers of all skill levels to use machine learning technology
Amazon Simple Notification Service: Amazon SNS is a fast, flexible, fully managed push notification service that lets you send individual messages or to
fan-out messages to large numbers of recipients. Send push notifications to mobile device users, email recipients or even send messages to other
distributed services. With Amazon SNS, you can send notifications to Apple, Google, Fire OS, and Windows devices, as well as to Android devices in China
with Baidu Cloud Push. You can use SNS to send SMS messages to mobile device users worldwide. Amazon SNS can also deliver messages to Amazon
Simple Queue Service (SQS), AWS Lambda functions, or to any HTTP endpoint.
Products - Examples
www.sayonetech.com 16
Developmental Aids
Amazon Web Services
MailgunMailgun is an online service that provides a set of powerful APIs that allow you to send, receive, track and store email effortlessly. You can control everything
through intuitive, RESTful APIs.
Fast and simple HTTP API or SMTP interface.
Server-side MIME assembly. No libraries required.
Live logs and webhook event notifications.
Everything is built and priced to scale with you.
Easy SMTP integration and a simple, RESTful API abstracts away the messy details of sending transactional or bulk email. Scale quickly, whether you need to
send 10 or 10 million emails.
Route and forward email directly into your app or inbox. Email parsing turns your emails into easy-to-digest structured data and spam filtering keeps out
unwanted emails.
Searchable logs ensures you know what is happening to your email while tags make it easy to A/B test and report on your data, and all via our webhooks.
Advanced email validation to increase conversion. jQuery plug-in enables you to integrate it quickly into your web forms.
www.sayonetech.com 17
Developmental Aids
SendGridSendGrid is a cloud-based transactional email delivery and management service. The service manages various types of email including shipping notifications,
friend requests, sign-up confirmations, and email newsletters. It also handles internet service provider (ISP) monitoring, domain keys, sender policy framework
(SPF), and feedback loops. Additionally, the company provides link tracking, open rate reporting. It also allows companies to track email opens, unsubscribes,
bounces, and spam reports. The company integrated SMS, voice, and push notification abilities to its service through a partnership with Twilio. SendGrid offers
a freemium version and a Lite Plan (pay-as-you-go), as well as three expanded levels of service: Essentials, Pro, and Premier.
SendGrid’s clients include Uber, Twilio, Foursquare, Pandora, and Airbnb.
SendGrid moves transactional mail between web applications and end users. For example, every time content is pinned or re-pinned in Pinterest, an email is
generated to let users know that is happening. Every time something is tagged in a photo, the application generates emails, which goes to the user of that app.
SendGrid sends over six billion emails per month on behalf of those customers. Web developers are good at building web applications. However, every web
application needs an email component. It is easy to set up your own, SMTP server, to send email on behalf of a web app. But one in five emails can go missing,
and the way the Internet hands off emails from web applications to the end user is complicated. SendGrid addresses the problem where an application
generates email to a human, where lots of algorithms are looking for spam and what to block before it gets to the inbox.
www.sayonetech.com 18
Developmental Aids
Campaign TrackerBy adding campaign parameters to your URLs, you can identify the campaigns that send traffic to your site. When a user clicks a referral link, these parameters
are sent to Analytics, so you can see the effectiveness of each campaign in your reports. For example, you could use campaign parameters to find out if more
users arrive at your download page in an app marketplace from an in-app ad or a search ad. You could also use campaign parameters to find out how many
users come to your web site from a link that was included in a monthly email newsletter.
Campaign Tracker supports Google’s campaign variables, which are known as “Source”, “Medium”, “Term”, “Content” and “Name”. When you have campaigns
that use these URL variables the values can be passed directly to your WordPress forms and sent through your forms in a hidden field.
Support for Google Campaign URL variables
Support for addition custom campaign variables (up to six)
Support for leading WordPress form plugins
Support for any number of forms to have tracking configured
Allows you to track and store Google’s GCLID variable (Google Adwords)
Provides referrer information – showing how the visitor arrived at your site
Works with caching plug-in and server side cached environments
New create tracking URLs inside the WordPress editor for each page or post
New store your favorite tracking values inside the plug-in
www.sayonetech.com 19
Developmental Aids
SummaryThe developmental aids discussed here undoubtedly bring about considerable savings in time and effort. There are several more out there. The net result is
that project teams are able to deliver optimum solutions to customers within budget and time to complete satisfaction. If you are interested in knowing more
about third party software for your projects, please contact us. We at SayOne have vast experience of using these for successful deliveries and will be happy to
assist you.
www.sayonetech.com 20
Developmental Aids
www.sayonetech.com
Made with in Kochi