filesystem maintenance · 2020. 9. 28. · quota enabling quotaon -v -a -g -u verbose all group...

Filesystem Maintenance

René Serral-Gracià Xavier Martorell-Bofill1

1Universitat Politècnica de Catalunya (UPC)

May 26, 2014

Introduction Filesystems Disk verification LVM Backups

Lectures

1 System administration introduction2 Operating System installation3 User management4 Application management5 System monitoring6 Filesystem Maintenance7 Local services8 Network services9 Security and Protection

10 Virtualization

R. Serral-Gracià, et. al Filesystem 2


Outline

1 IntroductionGoals

2 Filesystems

3 Disk verification

4 Logical Volume Manager (LVM)

5 Backups



Goals

Knowledge

FilesystemsBackup toolsBackup media

Abilities

Filesystem ampliationFilesystem verificationPerform and restore backups



Outline

1 Introduction

2 Filesystems

3 Disk verification


5 Backups



Filesystems (I)

FAT (FAT16) –> DOSSmall disks (< 4GB)File names 8+3

FAT32 (VFAT) –> Win95Larger disksLong filenamesPartial definition of soft-linksNo owner or file access privileges

exFATFAT32 ExtensionTheoretical maximum capacity of 64ZiB (512TiB real)

NTFS –> WinNT, XP, Vista, Windows 7Integrates ownership and privileges (create, modify,access...)Maps to Windows NT security model



Filesystems (II)

ext2UNIX FilesystemSoft/hard linksAccess privilegesLong filenames

ext3Adds journaling (eases error recovery)

reiserfsFiles and directories organized similarly to a databaseFeatures journalingVery efficient in small filesNo internal block fragmentation



Filesystems (III)

xfsjournalingDynamic i-node managementACLsVery large disk sizesFilesystem activity log

jfsjournalingDynamic i-node managementACLs and MAC (Mandatory Acess Control)Very large disk sizes



Filesystems (i IV)

ext464 bits addressing, improvements in journalingDelayed allocationExtents1 exbibyte (EiB) maximum size

btrfsExtentsOnline resizingOnline balancingOnline filesystem check



Journal filesystems

Journal: disk operation registryEases the recovery of the FS in case of crash or errorSlightly decrease in disk operations performance

Journal outside the buffer cacheJournal can be stored in another disk or partition

Ext3/4, reiserfs, JFS, XFS, NTFS, BTRFS have journal



Outline

1 Introduction

2 Filesystems

3 Disk verificationDisk fragmentationFilesystem ampliationDisk quota management


5 Backups



Disk verification (I)

Reason for errors

Hardware errorsPower shortageOperating system bugsAdministration errors

Incorrect machine shutdown

Never verify a filesystem while mountedHigh probability of disk corruptionVerification access skips the buffer cache and acts directluon the device



Disk verification (II)

Logical verification

Filesystem metadataDirectory structureLost data recovery

Directory lost+found

Physical recovery

Disk blocks with Input/output errorsCommand: badblocks



Disk fragmentation

Faster access to consecutive disk blocksFaster access to closer files within the diskFaster access Depending on the disk region

Zona mitja vs. extrems del disc



Filesystem ampliation

Install and configure the new diskPartitionor recycle existing ones. . .

Create filesystemDecide mountpointsTransfer the required data to the new partitionMount the partition

Update /etc/fstab

Maybe you have to reorganize existing directories/home→ /homeA + /homeB/home→ /home/students + /home/professors



Exercise

Planificar i definir possibles ampliacions dels segÃ14ents

directoris

/home

/usr/local

/var



Disk quota (I)

QuotaAbility to limit the amount of data a user (or user group) is ableto use in a filesystem (partition)

Requires

Support from the filesystemSupport from the kernel



Quota management (II)

Partition preparation

Mounted using options ’usrquota’ and/or ’grpquota’It can be done from /etc/fstab

/dev/sda9 /home ext4 defaults,usrquota,grpquota 1 1

quotacheck command to create the quota filesquotacheck -v -a -g -u -m

verbose all group user no-remount

Creates/aquota.user/aquota.group



Quota management (III)

Quota enablingquotaon -v -a -g -u

verbose all group user

Activates quota mechanisms, usually from /etc/init.d/

Quota disabling/sbin/quotaoff

Quota editing (edquota)Disk quotas for user xavim (uid 500):Filesystem blocks soft hard inodes soft hard/dev/sdb1 3 16 32 2 0 0

Data blocs and i-nodes quotaIt is not possible to edit the used blocks/inodes, but thelimits of the quota can be changed



Quota management (and IV)

Visualize quotas: quota -v

Disk quotas for user xavim (uid 500):Filesystem blocks quota limit grace files quota limit grace/dev/sdb1 32* 16 32 6days 2 0 0 -

∗ We are above the quotas, within the "hard" limit!!

"Grace period"Grace time where the user can reach the hard limit, it onlyraises warningsIf the grace period expires, then the system does not allowto go above the soft limit



Other maintenance tasks

Monitoring

Free space (df)Most systems reserve a (5%) of the space to be exclusivelyused by root

Occupied space (du)

Synchronization

Write to disk the modified bufferssyncUpdate daemon



Outline

1 Introduction

2 Filesystems

3 Disk verification


5 Backups



Logical Volume Manager (LVM) (I)

Physical Volumes (PV)

/dev/sda1 /dev/sdb1 /dev/sdb2

Volume Group

homeext3

swap / (root)ext4

usrbtrfs

Logical Volumes (LV)



Logical Volume Manager (and II)

High level abstraction of the disk spaceAggregates multiple physical partitions/disks

Allows to add more partitions to the volumeIt allows logical partitions within the volume

They can be assigned logical namesCustomized distribution among the physical volumes

ResizingMove

Example: /etc/fstab/boot /dev/sda1 ...swap /dev/vg00/swap .../ /dev/vg00/root .../home /dev/vg00/home .../usr /dev/vg00/usr ...



Outline

1 Introduction

2 Filesystems

3 Disk verification


5 BackupsFull BackupIncremental BackupReverse Incremental Backup



Backups

Data to copyUser data (home, mail, ... )Program data (BBDDs, CVS, web, ...)System configurationBinary?

Backup frequencyData confidenceData importance

Backup typesFull Backup (all)Incremental Backup (only changes)Reverse Incremental Backup (only changes)



Full Backup

Always copy all the dataFast to restoreLarge size

Sistema

backup

backup

backup

Temps



Incremental Backup

Only backup the changed filesAdvantages

Small size

It can be done in any media

InconveniencesSlower to restore

The first one is equal to a fullbackup

Do not create a large backup sequence

Sistema

backup total

backup

Restaura

cióbackup

Tem

ps



Reverse Incremental Backup

Everything is copied but in the former backup only thechanges are kept

Advantages

Fast to restore

Little space

InconveniencesOnly possible in randomaccess media

Sistema

=backup

backup

backup =Temps

Restaura

ció



Backups

Physical support

Floppy, disc, CD, tape,network. . .To consider:

Cost size ratioReliability

AvailabilityUsabilitySpeed

Copy location

Accident protectionFireproof boxesKeep some backups outside the company premisesStealing protection



Exercise

Define a backup policy (data to backup, backup type,frequency, device, compression, . . . ) for a multi-user serverwithin a company with:

500 Gb. disk and 80 usersMail

50Mb per userWeb pages

20 Mb per user100 Mb company web

Code repository10 GB distributed among 20 projectsOnly 5 active projects



Other considerations

When having different servers it is recommended toDefine specific backup machines

cheapereasy to administer

Tools: tar+rsync/ssh, amanda, bacula



Personal Homework

Task automationProgramming language: bash, perlInformation search: find, grep. . .


filesystem maintenance · 2020. 9. 28. · quota enabling quotaon -v -a -g -u verbose all group...

Documents