AFS

Per-File ACLs

Marc Dionne

TechnoConseil

Outline

• Introduction• History• Issues -Protocol and semantics• Issue - Implementation• Issues - Compatibility• Current status• What's next

Introduction

• AFS user/administrator/developer for 15 years

• Mainly involved in development over the past few years– Linux client

• kernel updates, bugs, some improvements

– Code cleanup efforts

• Looking for a more substantial and challenging contribution

Introduction

• AFS only allows ACLs to be set at the directory level– All files share this ACL

• Disadvantage compared to other local or network filesystems

• Users new to AFS expect ACLs to work on files• Need workarounds for common situations: dot

files, .mailrc, etc– Some files need to be in a common location,

but need different rights• Some demand for this feature in the community

History

• Had questions about the feasibility – in particular the impact on the client side

• Coded an initial prototype – june 2009 – over a weekend– Simplest implementation possible

• New special file for file ACLs, parallels small vnode index – 1 slot per file

• Existing ACL RPCs

• Encouraging - surprisingly stable and functional– Very few changes needed on the Unix client

• ... but some issues lurking• Discussions on afs3-std and other venues• Several revisions since then

Issues - Protocol and semantics

• Current AFS protocol does not specify ACL operations on files– Requires new RPCs: fetchACL and storeACL

• Inheritance semantics need to be defined– Are ACLs inherited, when, how

• Inherit until set• Inherit always

– How does an ACL change on a directory affect files• Aim for least surprise for current AFS users• Behaviour of client tools

– fs setacl, listacl– vos move, restore, etc.

Issues - Compatibility

• General– All combinations of current and new clients and

servers should interoperate reasonably– OK to restrict new functionality – limit access to files

that have an ACL with broader access than the parent– But not OK to expose files that file ACLs should make

unreadable

• Servers and clients need to be aware of the other side’s status– Use client and file server capability bits in OpenAFS– Capabilities exchange for Unix client recently merged

Issues - Compatibility

• Current clients can leak cached data– They assume directory rights apply to all files, but

rights may now vary per file– Scenario

• Users A and B can read directory D• File F has a file ACL that allows A to read, but not B• A reads F, data is brought into the cache• B tries to read F, cache manager assumes rights to D apply,

and happily returns data– Possible solution: artificially restrict rights

• Fiddle with the returned user rights on the server side, or the file mode bits

• Compute most restrictive rights for that user for all files within a directory – return the same rights for all files

Issues - Compatibility

• Windows– Tests showed the Windows client reacted badly with files

more restricted than the parent directory• Mainly lengthy hangs in explorer

– Commit 9e8ae43b introduced a registry key to correct this behaviour

• Should be activated based on server capabilities• Same solution should apply – return most restrictive rights in the

directory

• Volume moves and restores– Prevent data (ACLs) loss while moving volumes to an

older server– Allow forced moves

Issues - Implementation

• Changes are required to the existing on-disk structures• On-disk Vnode structure is full

– RXOSD already repurposes some elements (vnode “magic”)– Really need a pointer in the Vnode – alternatives are much more

complex (hashing, etc.)– Current scheme requires a power of 2 size

• Small vnode size would have to double• May be a concern for sites with large numbers of files

• Volume header is nearly full– RXOSD repurposes an existing file pointer

• New volume data (file ACLs) need to be preserved across volume clone, dump, restore and move operations– New dump tags

Current status

• Prototype implentation– Published as a github clone:http://github.com/mdionne/openafs

• New per volume special file for file ACLs– Reference counted entries, although 1 entry per file currently– Linked entries to track available slots

• Reuse “magic” field in on-disk vnode structure as an ACL pointer– known conflict with RXOSD

• In memory, file ACL follows the on-disk vnode structure (similar to directories)– ACL is stored and read along with the Vnode (VnLoad, VnStore)– ACL modification triggers vnode writeback

Current Status

• New fetchACL and storeACL RPCs defined and used– Identical signature to current RPCs– New clearACL RPC needed

• Client capability identifies file ACL support– Used to determine whether fileserver should restrict rights

• Rights restrictions not implemented yet• Some security concerns whether it's acceptable to rely on

capabilities to trigger this

• Server capability– Clients know not to assume that directory rights apply to all

files– Clients use new ACL RPCs

Current Status

• Inherit until set semantics– Once a file ACL is set, it is “detached” from the parent - ACL

changes to the parent will no longer affect it– New files have no file ACL – parent ACL applies– fetchACL returns a special value to indicate no ACL– New clearACL RPC to re-attach to parent ACL

• listacl– Show file ACL, or an indication that there is none

– Option to show effective ACL

• Volume dumps– New tag identifies a file ACL– ACL retrieved from special file and added to dump if needed– On restore, insert ACL into target volume special file

What's Next

• Getting consensus and document protocol changes and semantics (afs3-std)

• RPC refresh – new ACL RPCs• Consensus on on-disk structures and implementation,

particularly the possible Vnode expansion• Unimplemented features

– Restrict rights for legacy clients– Windows client changes– Volume manipulation safeguards– Documentation changes

• Goal: keep the scope under control– Better chances of getting it done and integrated in a

reasonable timeframe

Parting thoughts

• Code is easy - getting consensus is harder– Small number of key people– Few opinions, some disagreement

• File server code is more intuitive than cache manager code– And userspace code is easier to debug than a kernel

module– But bugs can be more painful...

• Dependencies and links with other pending work don't help– RPC refresh, out of tree projects (RXOSD)– Other ongoing or potential projects: alternate data

streams, extended attributes, etc.

Thank you

Questions or comments ?