grasp(eo) versioning system final presentation
DESCRIPTION
Master semester project presentationTRANSCRIPT
![Page 1: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/1.jpg)
Versioning system in Graspeo
Nikita Grishin, EPFL, 02.06.2014 Supervised by Andrii Vozniuk
![Page 2: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/2.jpg)
![Page 3: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/3.jpg)
Motivation
Every document management system has the versioning.
![Page 4: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/4.jpg)
Motivation
Every document management system has the versioning.
Various resources from various locations need to be tracked back
![Page 5: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/5.jpg)
Motivation
Every document management system has the versioning.
Various resources from various locations need to be tracked back
The collaborators system implies the recovery system
![Page 6: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/6.jpg)
State of the art
![Page 7: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/7.jpg)
• Mercurial, SVN/CVS!
• On document change, only modifications are saved
• Easy to track modifications
• Difficult to recover old versions
• Git
• On file change, new version of file is created
• Easy to recover old versions
• Difficult to track modifications
![Page 8: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/8.jpg)
• Mercurial, SVN/CVS!
• On document change, only modifications are saved
• Easy to track modifications
• Difficult to recover old versions
• Git
• On file change, new version of file is created
• Pointer to the last version
• Easy to recover old versions
• Difficult to track modifications
Only for text documents.
Works very bad with media content.
![Page 9: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/9.jpg)
• Snapshots: new version of document for each modification
![Page 10: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/10.jpg)
• Snapshots: new version of document for each modification
• Recovering the older version: creates a copy of the asked version on the top of the last version
![Page 11: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/11.jpg)
• Snapshots: new version of document for each modification
• Recovering the older version: creates a copy of the asked version on the top of the last version
• Some old versions can be removed by Google in case of lack of space
![Page 12: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/12.jpg)
• Snapshots: new version of document for each modification
• Recovering the older version: creates a copy of the asked version on the top of the last version
• Some old versions can be removed by Google in case of lack of space
• Only users that can edit the document can see its modification history
![Page 13: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/13.jpg)
• Hard drive snapshots.
Apple TimeMachine
![Page 14: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/14.jpg)
• Hard drive snapshots.
• Keeps:
• Hourly backups for last 24 hours
Apple TimeMachine
![Page 15: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/15.jpg)
• Hard drive snapshots.
• Keeps:
• Hourly backups for last 24 hours
• Daily backups for the past month
Apple TimeMachine
![Page 16: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/16.jpg)
• Hard drive snapshots.
• Keeps:
• Hourly backups for last 24 hours
• Daily backups for the past month
• Weekly backups until your backup drive is full.
Apple TimeMachine
![Page 17: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/17.jpg)
• Hard drive snapshots.
• Keeps:
• Hourly backups for last 24 hours
• Daily backups for the past month
• Weekly backups until your backup drive is full.
• When your backup drive is full, TimeMachine removes old backups to free space.
Apple TimeMachine
![Page 18: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/18.jpg)
Graspeo versioning system solution
Versioning
![Page 19: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/19.jpg)
Graspeo versioning system solution
Versioning
• Snapshots: new version of item for each significant modification
![Page 20: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/20.jpg)
Graspeo versioning system solution
Versioning
• Snapshots: new version of item for each significant modification
• Significant modification is a modification that needs to be stored in versions.
• Space: creation of space, modification in subitems list
• Resource: creation of resource, modification of its content (resource replacement)
![Page 21: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/21.jpg)
Graspeo versioning system solution• Snapshots: new version of item for each
significant modification
• Significant modification is a modification that needs to be stored in versions.
• Space: creation of space, modification in subitems list
• Resource: creation of resource, modification of its content (resource replacement)
• Restore a version remains to copy the content from version document to an item document. This will automatically create a new version on the top of versioning tree.
Versioning
![Page 22: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/22.jpg)
![Page 23: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/23.jpg)
How does it work?• New collection for versions: easy scalable
![Page 24: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/24.jpg)
How does it work?• New collection for versions: easy scalable
• Version document is almost a full copy of item document
![Page 25: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/25.jpg)
How does it work?• New collection for versions: easy scalable
• Version document is almost a full copy of item document
• Version creation happens in mongoose post-save hook
![Page 26: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/26.jpg)
How does it work?• New collection for versions: easy scalable
• Version document is almost a full copy of item document
• Version creation happens in mongoose post-save hook
• The versions controller takes a decision if a new version is needed (i.e. defining a notion of significant modification)
![Page 27: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/27.jpg)
How does it work?
• Resources version:
• GridFS to store old file versions
• The last version of file is also stored on local filesystem because of performance issues with GridFS and use of BTSync
![Page 28: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/28.jpg)
How does it work?
• Version Schema
• Extends Item schema
• New field called «originId» to track one item modifications
![Page 29: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/29.jpg)
How does it work?• Version is created on item creation and on
significant modification
![Page 30: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/30.jpg)
How does it work?• Version is created on item creation and on
significant modification
• Space:
• Modification of subitems list (subitem created/removed)
• Resource replacement
![Page 31: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/31.jpg)
How does it work?• Version is created on item creation and on significant
modification
• Space:
• Modification of subitems list (subitem created/removed)
• Resource replacement
• Resource:
• Replacement
![Page 32: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/32.jpg)
How does it work?
• Versioning API
• List of versions timestamps
• Version by date
• Restore a chosen version of space
![Page 33: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/33.jpg)
How does it work?• Version restore
• User asks to restore the version of space for given date
• System looks for that space in items collection and for its version in versions collection
• Copies subitems list from version to item document
• Removes all items in items collection that are no more in recovered space
• Recursively restore subitems of that space
![Page 34: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/34.jpg)
How does it work?• Version restore
• While recursively restoring subitems of space, some of them can be already removed from items collection. For that case:
• The system creates a new document in item collections and fill it with data from version document
• New document = new id: the system fix all path- and id-related problems
• That is the most expensive part of restoring process
![Page 35: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/35.jpg)
How does it work?
• Version restore: Resource
• Make a restore in database as described above
• Write the asked version of resource to disk
• Regenerate thumbnails for the new resource content.
![Page 36: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/36.jpg)
How does it work?• On the user side:
• Version creation is completely transparent
• Version restore is a little bit slow
• On the server side:
• Version creation is asynchronous and with a low load on database
• Version restore is expensive in terms of time and database load
![Page 37: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/37.jpg)
Future Plans
![Page 38: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/38.jpg)
Future Plans• Optimize the version restore process
• Extend the current versioning system to a multiuser versioning
• Integrate Git with current versioning system for text files
• Create a movie-style visualization of versioning and modification tracking
![Page 39: Grasp(eo) versioning system Final presentation](https://reader033.vdocuments.net/reader033/viewer/2022060123/559773451a28ab6c508b4585/html5/thumbnails/39.jpg)
Thank you for your attentionQuestions