dbsight scalability

16
DBSight Built To Scale! •No JVM Global Garbage Collection! •Fastest Batch Indexing •Fastest Search •Built-in Support for Index Replication •Built-in Distributed Shard Search

Upload: chrislusf

Post on 16-Jun-2015

2.349 views

Category:

Technology


1 download

DESCRIPTION

Not only DBSight is easy to use, but also DBSight scales very well!

TRANSCRIPT

Page 1: DBSight Scalability

DBSight Built To Scale!

• No JVM Global Garbage Collection!• Fastest Batch Indexing• Fastest Search• Built-in Support for Index Replication• Built-in Distributed Shard Search

Page 2: DBSight Scalability

No JVM Global Garbage Collection!

• Global GC is bad!• Global GC for 1 GB memory can be 10s

of seconds• During the long stop-the-world GC, no

search response at all!• Some customers are already running

20G memory

Page 3: DBSight Scalability

No JVM Global Garbage Collection!

• How to tune GC?• Mark and Sweep• Garbage First (G1)• …

• None works when index switches!• There is simply a lot of memory to

update• Lots of caches needs priming• Lots of old caches discarded

Page 4: DBSight Scalability

No JVM Global Garbage Collection!

• DBSight simply avoids the GC problem• DBSight servlet has efficient small memory

footprint• Indexing and searching are separated

processes.

Page 5: DBSight Scalability
Page 6: DBSight Scalability

No JVM Global Garbage Collection!

• DBSight simply avoids the GC problem• DBSight servlet has efficient small memory

footprint• Indexing and searching are separated

processes.• A new search process is created for each new

set of index files, and warmed up before serving the requests

Page 7: DBSight Scalability
Page 8: DBSight Scalability

No JVM Global Garbage Collection!

• DBSight simply avoids the GC problem• DBSight servlet has efficient small memory

footprint• Indexing and searching are separated

processes.• A new search process is created for each new

set of index files, and warmed up before serving the requests

• The new search process is “online”, the old search process is killed• Killing a process and start a new one is much faster

than going through gigabytes of memory

Page 9: DBSight Scalability
Page 10: DBSight Scalability

Fastest Batch Indexing

• Dedicated Indexing process does not interfere with other processes’ memory allocation.

• Batch indexing is much faster than online indexing

• Some open source approaches try to accept and index new content via HTTP, or during ORM updates.• Works for low updates site• Slow or Die when a lot of updates

• Not good if you need to re-create the index

Page 11: DBSight Scalability

Fastest Search

• Dedicated Search process does not interfere with other processes’ memory allocation.

• Dedicated memory space allows more caches• Batched index updates allow memory

optimization

Page 12: DBSight Scalability

Built-In Index Replication

• Even dedicated indexing process still competes on CPU, memory, disk

• DBSight supports dedicated indexing machine with several dedicated search machines.

Page 13: DBSight Scalability

Index Replication: 2-node setup

Page 14: DBSight Scalability

Index Replication: multi-node

Page 15: DBSight Scalability

Distributed Shard Search – Linear Scalability

Page 16: DBSight Scalability

DBSight Scales You Up!

• Adding search takes just mouse clicks• Powers you to tackle any database• Big tasks seem easy• Overdue search projects done in

minutes!