Automate Roadmap

Madhu KanoorAutomate Teammkanoor@redhat.com

Agenda

BackgroundRecent changesFuture PlanQ & A

BackgroundAutomate is a ManageIQ toolset used to

1. Extend the product to implement customer specific requirements

2. Update the behavior of the different workflows in the product

3. Integrate with other products

Examples

4. Set the name of a VM or instance to be based on specific enterprise guidelines.

5. Control the placement of VM’s or instances.

6. Custom buttons to enable common tasks e.g. Add Hard Drive to a VM

Object ModelAutomate uses an object model comprising of

● Domains● Namespaces● Classes● Instances● Methods

Domain

Namespace1 Namespace2

Class1 Class2 Class3 Class4

Instance1 Method1 Instance2 Method2

Domain Search

NAMESPACE1/CLASS1/INST1 will get picked from DOMAIN2NAMESPACE1/CLASS1/INST2 will get picked up from DOMAIN1This is a kin to the file search path when looking for executables in operating systems. (ls versus /bin/ls)

DOMAIN1 (Priority 10)NAMESPACE1

CLASS1● INST1● INST2

DOMAIN2 (Priority 20)NAMESPACE1

CLASS1● INST1● INST3

StorageAutomate object model is stored in a relational database in the following 6 tables

1. miq_ae_namespaces2. miq_ae_classes3. miq_ae_instances4. miq_ae_methods5. miq_ae_fields6. miq_ae_values

Basic Flow

PostgresSQL

Automate ExplorerRead/Write/Reset/Restore

Tools (Export/Import)

Automate EngineProcess Model

R/W

R/W

R

Recent ChangesIn the Anand release of ManageIQ the following changes were introduced

1. The automate model was converted to YAML fora. Seeding the databaseb. Exporting models from PostgresSQLc. Importing models into PostgresSQLd. Restoring the Automate model from a backup

2. Domains were introduced to segregate the modelsa. ManageIQ (Readonly)b. User/Customer (Read/write)c. Each domain is stored in a separate directory (vmdb/db/fixtures/ae_datastore)

3. Instance and Method overrides

Future Plan

Version control Automate Model

REST API for Automate Methods

Separate out the Automate Engine as a Ruby Gem/Rails Engine

Version Control Automate Model

Use cases

● Users would like to make incremental changes to the Automate model and revert back to a previous state.

● User edits are currently written directly to the database and cannot be easily undone.

● Audit Trail

Proposal

Use GIT to Version control the Automate modelThe Automate Engine reads the model from the GIT repositoryThe MIQ Automate Explorer will make changes to the Automate model, which will get saved via GIT.

Comparison

PostgresPros:Single database can be shared between multiple appliances in a zone.TransactionsBackups/ReplicationsCons:Lacks version control

GIT + Filesystem based DBPros: Git provides the version control Provides History Ability to undo changesShare repositoriesCons:Another system to be managed for backups, replications etc.

Multi Appliance Environment

Appliance 1

Appliance 2

Appliance 3

Appliance 4

GITMaster

GITSlave

GITSlave(Secondary)

GITSlave

Clone/Pull

Clone/Pull Clone/Pull

Domains

Domains

Domains

Domains

Postgres

ManageIQ

External Repos

Write Repo Metadata

GIT Master

Manages the domain repository information in Postgres

Internal Domains (Customer typically Read/Write)

External Domains (ManageIQ,Vendor1,Vendor2 Readonly)Repository LocationLast Commit Information

Automate Explorer Read/Write Automate ModelAll Import/Export runs on this appliance only

GIT Slave

SynchronizationReads the repository information from PostgresSQLDisables the Automate Worker role for the applianceWaits for all the existing Automate Workers to endSynchronizes the Automate DB (add/update/delete domains)Re-enables the Automate Worker role for the appliance

Automate Explorer read only mode

Commits

● Commit Boundarieso Single File changes (Instance, Method)o Multiple File Changes (Domain, Namespace, Class

changes)

Discussion Points

- GIT Server protocol (ssh, https?)- Should we allow for branching?- How often should we commit changes on

GIT Master?- How often should the GIT pull be run on the

connected appliances?

Discussion Points

● RBAC for Automate Model● UI controls● Single/Multiple appliances allow for editing the model● Undoing Commits

o Undoing back to a specific commito Undoing a single commit

Demo

Automate Explorer using the filesystem as a storage for Automate Model.

Comparison

PostgresSQL

ActiveRecord

UI UI

FileSystemAccess

GIT

Replacing ActiveRecord Classes

● ActiveModel● Validation● Attribute Methods● Naming

● MiqAeGit○ Implements the GIT Interface using rugged gem○ Cloning, Commit

● MiqAeModelBase○ Included by MiqAeNamespace, MiqAeDomain, MiqAeClass, MiqAeInstance,

MiqAeMethod○ Read/Write routed thru MiqAeGit

REST API Methods

REST API for Automate Methods

Automate methods are executed by the Automate Engine.Currently uses DRb to establish connection with the Method process to exchange ruby objects.Ruby is the only language supported because of DRb

Method Dispatch

The Automation Engine Worker

VMDBWorkspace

DRbServer Automate Method

DRb ClientService Model

Method Dispatch (REST)

The Automation Engine Worker

VMDB WorkspaceAutomate MethodREST Method

Web Service Worker(REST Server)

Write

Plan

Add a new aetype for REST based methodsThis would trigger saving of the workspaceGenerating a token for the workspace persisted to the DBThis has been implemented previously when we supported Perl and other languages.

Discussions

Persisting the workspaceReading/Writing workspace from the REST WorkerReconstituting workspace for the Automate Worker when the method ends

Automate Engine Gem

Automate Engine Gem

Automate Engine is responsible for- Reading the Automate Model- Instantiating a workspace to store objects- Compose in memory objects from the

automate model- Execute Methods- Execute State Machines

Advantages of a Gem

● Easier to maintain● Modular● Isolated testing

Discussions

Dependency on core class in VMDB (MiqQueue)

Deliver from QueueQueue for retriesExposing the Service ModelRegistering built in methods that don’t use DRb

Q & A