Perforce Merge 2013

Gopal Thakkar-Application Engineer, Bangalore

• Market Drivers

• Technology approaches

• IC Manage Views - Live demo

Workspace Acceleration & Network Storage Reduction

Driven by Semiconductor Design, Video game software, Enterprise software…

Perforce depots: multi-TB+, 100,000,000s+ files

Single workspaces: multi-GB, 10,000s+ files

Users: 1000’s across globe

Digital Assets Growing at Rapid Rate

Application Slowdown - ave. 30% tool iteration slowdown

Source: 2012 LaunchM Survey, 524 respondents

Storage issues - costs + capacity not keeping up

Workspace syncs slow - remote and/or local

Impact of Increasing Digital Assets: Survey cites Top File System Problems

Accelerate

workspace syncs

Remove network

bandwidth bottleneck

Reduce network disk

space usage

SLOW Syncs

Vision

Network Storage

Network Storage

Workspace C File 1 File 2 File 3

Workspace B File 1 File 2 File 3

Workspace A File 1 File 2 File 3

User A

User B

User C

• Duplicates file storage for every user workspace

• De-dupe too slow for fast changing data sets

• Saturates precious Tier 1 storage space

• Under-utilizes local caching

High Demand on Network Storage

Perforce

SCM Server

File Data

Metadata

File 1 File 2 File 3

File Not Yet

Opened

User A

User B

User C Local Cache

File 2

0 b

File 1

0 b

File 3

0 b

• Virtual workspaces projected using Perforce db.have table information from fstat

• 100% dynamic, user controlled workspaces

• Local storage agnostic, e.g. disk, tmpfs, SSD....

Local Cache

File 2

0 b

File 1

0 b

File 3

0 b

Local Cache

File 2

0 b

File 1

0 b

File 3

0 b

Dynamic Virtual Workspaces Instantly displays fully populated workspaces

Local Caching Streams actual files on application / user demand

User A

User B

User C

Local Cache

File 2

0 b

File 1

0 b

File 3

0 b

• Files can be streamed from main Perforce server, Perforce proxy or Perforce replica

• TCP streaming protocol avoids traditional NFS-delivery latency

• 2x faster retrieval of cache-warm files due to on-board access

File Not Yet

Opened

File Opened

Read-only

Local Cache

File 1

0 b

File 3

0 b File 2

Local Cache

File 2

0 b

File 1

0 b File 3

User A

User B

Local Cache

File 2

Local Cache

File 3

0 b

File 2

0 b

User C Local Cache

File 2

0 b

File 1

0 b File 3

File 1

0 b

Automatic cache management stays within storage quotas using LRU algorithm

File 3

File

File 1

File Not Yet

Opened

File Opened

Read-only

Local caching - manages cache quotas

Intelligent File Redirection: 2x faster file access

Reads

Writes

Intelligent File Filtering approach •Separates reads from writes, stores reads in local cache •Modified files automatically written to NAS (or local cache) Advantages •Gets on-board speeds for reads instead of network storage •Modified files safely stored on network disks •Automated - No manual management

Intelligent File Redirect On-board speeds for reads

Network Storage

Local Cache

Read/

Write

Read

-only

Read

-only

Read

-only

Read/

Write

Read/

Write

User A

User B

Local Cache

Local Cache

File 3

0 b

File 2

0 b

User C Local Cache

File 2

0 b

File 1

0 b File 3

File 1

0 b

Newly created files, checked out files saved to network storage volume or kept local

File 3

File

File 2

0 b

Network Storage

TEMP 1

File Opened

Read/Write

File 1

File Not Yet

Opened

File Opened

Read-only

Intelligent file redirect

User A

User B

Local Cache

Local Cache

File 3

0 b

File 2

0 b

User C Local Cache

File 2

0 b

File 1

0 b File 3

File 1

0 b

Checked in objects automatically purged from write storage

Avoids duplicated files, frees up space.

File 3

File

File 2

0 b

Network Storage

File 1 TEMP 1

Real-time de-duplication - frees space after Perforce check-ins

File Opened

Read/Write

File Not Yet

Opened

File Opened

Read-only

IC Manage Views: Accelerates Workspace Syncs & Reduces Network Storage

IC Manage Views

Network storage usage reduced

by 4x.

Local Caching,

Real-time De-dup

Zero-Time Sync Workspaces.

Dynamic Virtual Workspaces

2x faster file access,

application speed up.

Intelligent File Redirection

IC Manage Views Additional Benefits

• NAS agnostic - 100% compatible with existing storage technologies

• Scalable - Savings increase with number of users and the size of databases.

• Flexibility to build Workspaces on Demand. Software teams can build workspaces anywhere, avoiding problems and costs associated with disk space allocation.

• Reliable. Handles cache recovery in the event of failures or errors.

• Designers maintain workspace file control and stability. No manual management of network cache storage and different versions

• Reduces load on filers and network.