dat300 notification services programming: introduction
DESCRIPTION
DAT300 Notification Services Programming: Introduction . Tete Mensa-Annan Program Manager SQL Server Microsoft Corporation. Focus of this session. Goal: Explain Notification Services concepts and features Non-Goal: Teach how to develop Notification Applications. Agenda. - PowerPoint PPT PresentationTRANSCRIPT
DAT300DAT300
Notification Services Notification Services Programming: Introduction Programming: Introduction
Tete Mensa-AnnanTete Mensa-AnnanProgram ManagerProgram ManagerSQL ServerSQL ServerMicrosoft CorporationMicrosoft Corporation
Focus of this sessionFocus of this session
• Goal:Goal:– Explain Notification Services concepts and Explain Notification Services concepts and
featuresfeatures
• Non-Goal:Non-Goal:– Teach how to develop Notification ApplicationsTeach how to develop Notification Applications
AgendaAgenda
• What is SQL Server Notification ServicesWhat is SQL Server Notification Services– IntroductionIntroduction
• ArchitectureArchitecture– Under the hoodUnder the hood
• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts
• DeploymentDeployment– ScenariosScenarios
What is the SQL Server What is the SQL Server Notification Services Platform?Notification Services Platform?• A platform for developing and deploying highly A platform for developing and deploying highly
scalable, rich Notification Applicationsscalable, rich Notification Applications
• A rich yet simple declarative model used to describe Notification A rich yet simple declarative model used to describe Notification ApplicationsApplications
• An efficient and scalable server, based on SQL Server 2000 An efficient and scalable server, based on SQL Server 2000 and .NET Framework, that execute Notification Applicationsand .NET Framework, that execute Notification Applications
– You write Notification Application - Notification Services You write Notification Application - Notification Services hosts and executes ithosts and executes it
Notification ApplicationsNotification Applications
A new class of scalable applications A new class of scalable applications that enable new scenarios delivering that enable new scenarios delivering personalizedpersonalized and and timelytimely information information to to any device.any device.
PersonalPersonal
Any deviceAny device
TimelyTimely
Notification ApplicationsNotification Applications
Events(thousands)
Notifications(millions)
Match
Subscriptions(millions)
Activity Generation Delivery Receipt
SQL Server on Windows,Oracle on Solaris,DB2 on S/390.
Custom Data Store.
SQL Server 2000 Notification Services
Exchange Server,WinRTC,SMS Aggregator,
Custom Mechanism.
Outlook,XP Messenger,Pocket PC,SmartPhone 2002,SMS capable phone,
Custom Mechanism.
Information flow
Generation vs. DeliveryGeneration vs. Delivery
Why Notification Services?Why Notification Services?
• Build and deploy application quicklyBuild and deploy application quickly(Time To Market)(Time To Market)– You customize within Notification Services frameworkYou customize within Notification Services framework– You focus on application, not platform designYou focus on application, not platform design
• RichnessRichness– TimeZones, Recurrence, Fall behind, RetryTimeZones, Recurrence, Fall behind, Retry
• PerformancePerformance– SQL matches Events and Subscriptions efficiently SQL matches Events and Subscriptions efficiently – A true ‘Internet’ scale solutionA true ‘Internet’ scale solution
AgendaAgenda
• What is SQL Server Notification ServicesWhat is SQL Server Notification Services– IntroductionIntroduction
• ArchitectureArchitecture– Under the hoodUnder the hood
• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts
• DeploymentDeployment– ScenariosScenarios
Subscription Management ApplicationSubscription Management Application
SubscriptionsSubscriptionsSubscribersSubscribers
CustomCustom
Eve
nt
Eve
nt
Co
llector
Co
llector
Event Provider HostEvent Provider Host
File SystemFile System
SQL ServerSQL Server
NotificationNotificationGeneratorGenerator
SQL MatchSQL MatchRulesRules CustomCustom
FormatterFormatter
DistributorDistributorSMSSMS
EmailEmail
FileFile
CustomCustom
EventsEvents
NotificationsNotificationsEvent BatchesEvent Batches
Notification BatchesNotification Batches
Subscriber Subscriber DevicesDevices
App. specificApp. specific
StandardStandard
XSLT XSLT FormatterFormatter
Architecture OverviewArchitecture Overview
Architecture DetailsArchitecture Details
Subscription Management Subscription Management ApplicationApplication
Subscription Management ApplicationSubscription Management Application
• Allows subscribersAllows subscribersto manage subscriptionsto manage subscriptions
• Can be written in:Can be written in:– Managed code: ASP .NET, Managed code: ASP .NET,
C#, VB.Net, etcC#, VB.Net, etc
– Unmanaged code: ASP, Unmanaged code: ASP, C++, VB6, etcC++, VB6, etc
• Deal with three classesDeal with three classes– SubscriberSubscriber• Represents Represents whowho gets informationgets information• Usually one entry per personUsually one entry per person
– SubscriptionSubscription• Represent Represent what & whenwhat & when information is sentinformation is sent• Multiple subscriptions per subscriberMultiple subscriptions per subscriber
– SubscriberDeviceSubscriberDevice• Represents Represents wherewhere to send the information to send the information• Multiple devices per subscriberMultiple devices per subscriber
Subscription Management ApplicationSubscription Management Application
Architecture DetailsArchitecture Details
Event ProvidersEvent Providers
Event ProvidersEvent Providers
• Event Providers collect data changes and submits them as Event Providers collect data changes and submits them as Events in Event BatchesEvents in Event Batches
• Notification Applications have one or more Event ProviderNotification Applications have one or more Event Provider
• Event Providers map to Event ClassesEvent Providers map to Event Classes
• Notification Services ship with two standard providersNotification Services ship with two standard providers
– File System Watcher Event ProviderFile System Watcher Event Provider– SQL Server Event ProviderSQL Server Event Provider
Custom Event ProvidersCustom Event Providers
• Hosted Event ProviderHosted Event Provider– Run in-proc with Notification Services Windows ServiceRun in-proc with Notification Services Windows Service– Must implement IEventProvider or IScheduledEventProvider interfacesMust implement IEventProvider or IScheduledEventProvider interfaces– Less custom code by leveraging Initialize, Run, Terminate, ScheduleLess custom code by leveraging Initialize, Run, Terminate, Schedule– Starts and Stops with Notification Services Windows ServiceStarts and Stops with Notification Services Windows Service
• Non-Hosted Event ProviderNon-Hosted Event Provider– Can run on alternate platform or remote systemCan run on alternate platform or remote system– Starts and Stops independently from Notification Services Windows ServiceStarts and Stops independently from Notification Services Windows Service
• Event Submission MethodsEvent Submission Methods– Event ObjectEvent Object– XMLLoader ObjectXMLLoader Object– Event Submission SQL Server Stored ProceduresEvent Submission SQL Server Stored Procedures
• A Custom Event ProviderA Custom Event Provider– Accepts event data from an external sourceAccepts event data from an external source– Creates and populates an event object with event dataCreates and populates an event object with event data– Calls Event Collector “Write” for each eventCalls Event Collector “Write” for each event– Calls Event Collector “Commit” to submit the batch of eventsCalls Event Collector “Commit” to submit the batch of events
EventsEventseventseventsWriteWriteWriteWriteWriteWrite
CommitCommitProvider HostProvider Host
Ev
ent
Ev
ent
Co
llecto
rC
ollec
tor
CustomCustomEvent ProviderEvent Provider
EventEventIBMIBM 102 ½102 ½MSFTMSFT 65 ¼65 ¼ORCLORCL 8 ¼ 8 ¼
IBMIBM 102 ½102 ½
MSFTMSFT 65 ¼65 ¼
ORCLORCL 8 ¼8 ¼
Event Object SubmissionsEvent Object Submissions
XMLLoader Object SubmissionsXMLLoader Object Submissions
• A Custom Event ProviderA Custom Event Provider– Accepts event data from an external sourceAccepts event data from an external source– Creates an XMLLoader object; optionally initialized with SQL Creates an XMLLoader object; optionally initialized with SQL
annotated XML event schemaannotated XML event schema– Called with an XML document or stream Called with an XML document or stream
• Batch size = the number of events within the document or streamBatch size = the number of events within the document or stream
EventsEventseventsevents
LoadXMLLoadXML
Provider HostProvider Host
XM
LX
ML
Ev
ent
Ev
ent
ProviderProvider
Event Submission SQL Server Stored Event Submission SQL Server Stored ProceduresProcedures
Un-Hosted Custom SQL Server Stored Procedures Provider:Un-Hosted Custom SQL Server Stored Procedures Provider:
DECLARE @eventQuery DECLARE @eventQuery DECLARE @postQueryDECLARE @postQuery
SET @eventQuery = SELECT stocksymbol, stockprice from RDT1SET @eventQuery = SELECT stocksymbol, stockprice from RDT1UNION SELECT stocksymbol, stockprice from RDT2UNION SELECT stocksymbol, stockprice from RDT2
SET @postQuery = DELETE RDT1 DELETE RDT2SET @postQuery = DELETE RDT1 DELETE RDT2
EXEC NSEventSubmitBatchStockEvent @providerName, @eventQuery, @postQueryEXEC NSEventSubmitBatchStockEvent @providerName, @eventQuery, @postQuery
EventsEvents
SubmitSubmit
SQ
L S
tored
SQ
L S
tored
Pro
cP
rocProviderProvider
Raw DataRaw DataTable 1Table 1
Raw DataRaw DataTable 2Table 2
Architecture DetailsArchitecture Details
GeneratorGenerator
GeneratorGenerator• Matches events and subscriptions to produce Matches events and subscriptions to produce
notificationsnotifications• You write “match rules”You write “match rules”
with full power of SQLwith full power of SQL(Transact-SQL, sp, transactions…)(Transact-SQL, sp, transactions…)
MatchMatchRuleRule
GeneratorGenerator
SELECT Notify(SELECT Notify(S.SubscriberId, S.SubscriberId, S.DeviceName,S.DeviceName,S.Locale, S.Locale, S.Symbol, S.Symbol, E.Price) E.Price)
FROM FROM StockEvents E, StockEvents E, StockSubscriptions S StockSubscriptions S
WHERE WHERE E.Symbol = S.Symbol AND E.Symbol = S.Symbol AND E.Price >= S.TriggerPriceE.Price >= S.TriggerPrice
SubscribersSubscribers
Stock Stock SubscriptionsSubscriptions
Subscriber Subscriber DevicesDevices
Stock Stock EventsEvents
Stock Stock NotificationsNotifications
Rule ClassificationsRule Classifications
• Notification generation rulesNotification generation rules– Subscription Event rulesSubscription Event rules
• Multiple per Event ClassMultiple per Event Class• Multiple per Subscription ClassMultiple per Subscription Class• Notify when MSFT stock price Notify when MSFT stock price
goes above 70.00goes above 70.00
– Subscription Scheduled rulesSubscription Scheduled rules• Multiple per Subscription ClassMultiple per Subscription Class• TimeZoneTimeZone• RecurrenceRecurrence• Send MSFT stock price at 1PM Send MSFT stock price at 1PM
Mon-FriMon-Fri
• Chronicle state change rulesChronicle state change rules– Event Chronicle RuleEvent Chronicle Rule
• One per event classOne per event class• Used to maintain Event state Used to maintain Event state
like Latest stock priceslike Latest stock prices
– Subscription chronicle updatesSubscription chronicle updates• (usually embedded within (usually embedded within
Event & Scheduled Event & Scheduled Subscription Rules)Subscription Rules)
• Last time document changes Last time document changes were sent to subscriberwere sent to subscriber
Rule Firing SequenceRule Firing Sequence• Rules firingsRules firings
– Event Chronicle Rules Fired firstEvent Chronicle Rules Fired first– Subscription Event Rules fired second (Subscription Chronicle Updates)Subscription Event Rules fired second (Subscription Chronicle Updates)– Subscription Scheduled Rules fired third (Subscription Chronicle Updates)Subscription Scheduled Rules fired third (Subscription Chronicle Updates)
• Two in-order application processing options:Two in-order application processing options:– Quantum orderingQuantum ordering
• All event batches for quantum processed firstAll event batches for quantum processed first• All scheduled subscriptions processed secondAll scheduled subscriptions processed second• Never do more work than one Quantum worth at a timeNever do more work than one Quantum worth at a time
– Strict event batch orderingStrict event batch ordering• Scheduled subscriptions due before event batch arrival are processed firstScheduled subscriptions due before event batch arrival are processed first• Event batch is processedEvent batch is processed• Scheduled subscriptions due after event batch arrival are processed until end of Scheduled subscriptions due after event batch arrival are processed until end of
QuantumQuantum
Generator Falling BehindGenerator Falling Behind
• Generator schedule = QuantumGenerator schedule = Quantum
• Application configurable Quantum limitsApplication configurable Quantum limits– Event Chronicle Rule firing limitEvent Chronicle Rule firing limit
• Don’t process stock event batches older than 24hrsDon’t process stock event batches older than 24hrs• Don’t process news stories older than 12 hoursDon’t process news stories older than 12 hours• Don’t process traffic events older than 2 hoursDon’t process traffic events older than 2 hours
– Subscription Rule firing limitSubscription Rule firing limit• Don’t fall behind more than 15 minutes for basketball notificationsDon’t fall behind more than 15 minutes for basketball notifications• Don’t fall behind more than half hour on weather notificationsDon’t fall behind more than half hour on weather notifications• Don’t fall behind more than 20 minutes on traffic eventsDon’t fall behind more than 20 minutes on traffic events
Architecture DetailsArchitecture Details
DistributorDistributor
DistributorDistributor
• Formats and delivers NotificationsFormats and delivers Notifications– Transforms Notification using XSLT or Custom Content Transforms Notification using XSLT or Custom Content
Formatter moduleFormatter module– Applies device, country and region metadata to format Applies device, country and region metadata to format
the notification body and envelopethe notification body and envelope– Sends the Notification via a delivery protocolSends the Notification via a delivery protocol
NotificationsNotifications
DistributorDistributor SMSSMS
SMTPSMTP
SMSSMS
CustomCustom
ContentContentFormattingFormatting
Distributor OverviewDistributor Overview
• Generator Creates NotificationsGenerator Creates Notifications– Per Notification ClassPer Notification Class– Raw notifications as rows in SQL tablesRaw notifications as rows in SQL tables
• Distributor LoopDistributor Loop– Periodically scan for notification batches ready to be Periodically scan for notification batches ready to be
distributeddistributed– Format raw SQL notification data Format raw SQL notification data – Assemble formatted notifications into protocol packetsAssemble formatted notifications into protocol packets– Deliver the protocol packetDeliver the protocol packet– Handle delivery failuresHandle delivery failures
Content FormattingContent Formatting
• Format raw SQL Notification dataFormat raw SQL Notification data– Language-ContryRegion and DeviceType Language-ContryRegion and DeviceType
• Specified per NotificationClassSpecified per NotificationClass– Stock notifications are formatted differently from Flight notificationsStock notifications are formatted differently from Flight notifications
• Built-in XSLT Content FormatterBuilt-in XSLT Content Formatter– Applies a XSLT to NotificationsApplies a XSLT to Notifications
• Custom Content FormatterCustom Content Formatter– Must implement IContentFormatterInterfaceMust implement IContentFormatterInterface– Initialize, ContentFormat, Close methodsInitialize, ContentFormat, Close methods
Digest and MulticastDigest and Multicast
• Digest: several individual notifications for the same subscriber are Digest: several individual notifications for the same subscriber are combined into one combined into one – E.g. Stock notifications for MSFT, IBM, CSCO,… for a single subscriber are E.g. Stock notifications for MSFT, IBM, CSCO,… for a single subscriber are
combined to be sent as one notificationcombined to be sent as one notification– Enabled per notification classEnabled per notification class– Customer Satisfaction Customer Satisfaction
• Multicast: same notification sent to multiple subscribers Multicast: same notification sent to multiple subscribers – E.g. Weather report for Seattle is the same for all recipients E.g. Weather report for Seattle is the same for all recipients – Enabled per notification class Enabled per notification class – Formatting is done onceFormatting is done once– Underlying protocol can choose to physically send one message, or to send each Underlying protocol can choose to physically send one message, or to send each
message separatelymessage separately– Huge performance gain for distributor Huge performance gain for distributor
Retry LogicRetry Logic
• Handling delivery failures: If a delivery operation fails, the Handling delivery failures: If a delivery operation fails, the distributor can make multiple attempts distributor can make multiple attempts
• The developer specifies the retry schedule per notification class The developer specifies the retry schedule per notification class (per protocol)(per protocol)– Series of delay intervals Series of delay intervals
• Retry attempts will continue until the entire retry schedule has Retry attempts will continue until the entire retry schedule has been completed, or until the notification expiresbeen completed, or until the notification expires
• Notification is flagged as done and failed if all retry attempts failNotification is flagged as done and failed if all retry attempts fail– Vacuuming cleans up both successful and failed notificationsVacuuming cleans up both successful and failed notifications
AgendaAgenda
• What is SQL Server notification ServicesWhat is SQL Server notification Services– Marketing Fluff IntroductionMarketing Fluff Introduction
• ArchitectureArchitecture– Under the hoodUnder the hood
• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts
• DeploymentDeployment– ScenariosScenarios
Notification Services InstanceNotification Services Instance• An Instance hosts one or more Notification Applications that An Instance hosts one or more Notification Applications that
share Subscribersshare Subscribers
• Each Instance runs as a multithreaded Windows ServiceEach Instance runs as a multithreaded Windows Service– Hosts Event Providers, Generator, Distributor Hosts Event Providers, Generator, Distributor
• Multiple Instances per computerMultiple Instances per computer– Test and productionTest and production– Separate Windows Services,Separate Windows Services,
Databases, trust boundariesDatabases, trust boundaries
• Instance may span multipleInstance may span multiplecomputerscomputers– E.g., multiple DistributorsE.g., multiple Distributors NewsStock Sports
Subscribers
Internet01
Notification ApplicationNotification Application
• An Notification Application definesAn Notification Application defines– One or more Event Classes, Subscription Classes (Rules), and One or more Event Classes, Subscription Classes (Rules), and
Notification ClassesNotification Classes
• Each Notification Application is a separate SQL Server Each Notification Application is a separate SQL Server databasedatabase– Event, Subscription, Chronicle and Notification dataEvent, Subscription, Chronicle and Notification data
• System ElementsSystem Elements– Event Providers (input), Generator (rule evaluation), Distributors Event Providers (input), Generator (rule evaluation), Distributors
(output)(output)
Notification Services VersioningNotification Services Versioning
• Different Notification Services Instances running on the Different Notification Services Instances running on the same system may run different versions of Notification same system may run different versions of Notification ServicesServices
• Each Notification Services Instance must run the same Each Notification Services Instance must run the same version across all computersversion across all computers
• Multiple versions of Notification Services may be Multiple versions of Notification Services may be installed side-by-sideinstalled side-by-side
• Installation of a new version does not remove or Installation of a new version does not remove or replace existing versionsreplace existing versions
AgendaAgenda
• What is SQL Server notification ServicesWhat is SQL Server notification Services– Marketing Fluff IntroductionMarketing Fluff Introduction
• ArchitectureArchitecture– Under the hoodUnder the hood
• Instances, Applications, VersioningInstances, Applications, Versioning– Important conceptsImportant concepts
• DeploymentDeployment– ScenariosScenarios
Core Deployment ScenariosCore Deployment Scenarios
SQL / NS SQL NS
Availab
ility
NS NS
SQL SQL
SQLNS
SQLNS
Scenario 1 Scenario 2
EP
GEN
DIST
SQL
Scenario 3
Scenario 4 Scenario 5
Scalability
SecuritySecurity
• Designed with Security in mindDesigned with Security in mind– Windows AuthenticationWindows Authentication– SQL Server AuthenticationSQL Server Authentication
• Typical Accounts (low privileges)Typical Accounts (low privileges)– Notification Services Windows Service AccountNotification Services Windows Service Account
• Domain\NSService + SQL Server NSService LoginDomain\NSService + SQL Server NSService Login
– SQL Server Service AccountSQL Server Service Account• Domain\SQLServiceDomain\SQLService
– ASPNET AccountASPNET Account• Domain\ASPNET + SQL Server ASPNET LoginDomain\ASPNET + SQL Server ASPNET Login
• DBA AccountDBA Account Operations AccountOperations Account• Domain\DBADomain\DBA Domain\OpsDomain\Ops
SummarySummary
• Notification Services is a platformNotification Services is a platform
• Enables rapid development of robust scalable Enables rapid development of robust scalable Notification ApplicationsNotification Applications
• Easy to use Declarative Programming ModelEasy to use Declarative Programming Model
• A True .Net Enterprise Server ProductA True .Net Enterprise Server Product
Questions and AnswersQuestions and Answers
Don’t forget to complete the Don’t forget to complete the on-line Session Feedback form on-line Session Feedback form on the Attendee Web siteon the Attendee Web site
https://web.mseventseurope.com/teched/https://web.mseventseurope.com/teched/